Service spotlight: AWS Step Functions for real workflows

Every few months I see a team reinvent Step Functions badly: a Lambda that invokes another Lambda, wrapped in a try/except that retries three times and then writes a row to DynamoDB so a cron job can poll for stuck items. That’s a state machine with extra steps — literally. Step Functions is AWS’s managed orchestrator for exactly this shape of problem, and it’s worth knowing precisely when it earns its keep and when it’s overkill.

What it actually is

A Step Function is a JSON (or YAML, via the newer workflow studio) state machine defined in Amazon States Language. Each state does one thing — invoke a Lambda, call another AWS service directly via an “SDK integration,” branch on a condition, wait, fan out over a list, or hand off to a human approval step — and the service itself handles the transitions, retries, timeouts, and error handling between them. Execution history is retained and visualized automatically, so when something fails at 2am you get a diagram with a red X on the exact state that broke, not a pile of CloudWatch Logs you have to stitch together by request ID.

There are two flavors, and picking the right one matters:

• Standard workflows are built for long-running, auditable processes. Exactly-once execution, up to a year of runtime, full execution history retained in the console. Priced per state transition.
• Express workflows are built for high-volume, short-duration work (under 5 minutes) — think per-request orchestration behind an API. At-least-once execution, no persistent execution history in the console (it goes to CloudWatch Logs instead, which you pay for separately), priced per invocation duration and memory, closer to Lambda’s pricing model.

Using Standard for a workflow that fires 50,000 times a day doing simple API-to-API glue work is the single most common way people get an unpleasantly large Step Functions bill.

When to reach for it

• Multi-step processes with real failure modes — order fulfillment, video transcoding pipelines, ML training/inference chains, anything with a “do A, then B, and if B fails, do C instead of D” shape. Encoding that in nested Lambda try/except blocks gets unreadable fast; a state machine makes the actual business logic visible as a diagram.
• Fan-out/fan-in work. The Map state runs a step over every item in an array — in Distributed Map mode, up to 10,000 concurrent child executions reading directly from S3 or a JSON array, without you writing a single line of concurrency-control code.
• Long-running processes that need to survive restarts. A Standard workflow waiting on a human approval, an external webhook, or a batch job that takes six hours doesn’t cost you anything while it’s waiting — Step Functions isn’t polling, it’s holding state and will resume the instant a callback token comes back.
• Direct AWS SDK integrations. Step Functions can call over 200 AWS services’ APIs directly from a state definition — start a Glue job, put an item in DynamoDB, publish to SNS — with no Lambda in between. That’s one less function to deploy, monitor, and patch for what’s really just a passthrough API call.

When NOT to reach for it

If your “workflow” is two steps with no meaningful failure branching — call API A, then call API B with A’s result — you don’t need a state machine, you need a Lambda function or even just synchronous code in your existing service. The overhead of authoring, deploying, and versioning a state machine definition isn’t worth it for something a single function handles in ten lines. And for very high-throughput, sub-second, simple orchestration (think: per-request routing in a hot path), Express workflows can work, but you’re often better served by keeping that logic in application code and reserving Step Functions for where the auditability and visual execution history actually pay off.

The pricing gotcha

Standard workflows charge per state transition, not per execution — and transitions add up faster than people expect. A Map state iterating over 1,000 items, each running three sequential states, is 3,000 transitions in one execution, not one. At $0.025 per 1,000 transitions, that’s cheap in isolation, but a workflow that fans out over large datasets on a frequent schedule can quietly become one of the more expensive things in an account. The fix isn’t to avoid Map — it’s to use Distributed Map, which counts child workflow executions differently and is built for exactly this high-fan-out case, plus to actually look at the “state transitions” line item in Cost Explorer before a workflow goes from a proof of concept to a production schedule running every five minutes.

A practical tip

Use ResultSelector and OutputPath inside state definitions to trim what gets passed downstream instead of piping entire upstream payloads (including that giant DynamoDB item or Lambda response) through every subsequent state. Standard workflow execution history and each state’s input/output are capped at 256 KB — passing bloated payloads through unfiltered is the most common way people hit that limit and get a cryptic States.DataLimitExceeded error on a workflow that’s otherwise working fine.