PayTestForge: Repeatable Stripe Webhook & Failure Simulator for Indie SaaS

How do you test payment failures / weird edge cases in your app?

How do you test payment failures / weird edge cases in your app?

I ran payments engineering at a public SaaS company a few years ago. You'll want to automate as much of this as possible, and mix in occasional human-driven testing. How? - At the lowest level, unit tests. Ensure your code can handle expected failure scenarios (card decline, processor failure, issuer failure, etc) while not corrupting data (i.e., charge happens but is not stored in database). Code coverage tools are your friend (i.e. JaCoCo in Java) to identify missed branches. You can aim for a certain percentage, but that can be a pain, better to focus first on highest risk areas (i.e., charging a card). - One level up, end to end tests. We had a few tiers: - one which flexed the internal API endpoints to handle billing (i.e., it would create test customers, submit charges, wait a bit, verify all webhooks were received and handled, etc) - another which flexed the UI itself (do we show an error message when a card was declined, etc). - Reconciliation jobs - we'd ingest transaction settlement reports every day. The processor _said_ a payment was successful when we submitted it a few days ago, but that's not really true until the transaction settled (i.e., the money is in our account). Transactions that don't settle got marked as "open" and we'd kick off a workflow from there. - "Lateness" checks - we submitted a payment, redirected the user to the issuer to complete authorization, but they never returned. After 30m, time out the payment (mark as failed, allow user to retry at a later date). For tests, we had a few tricks to make these a bit less painful: - All tests ran in a non-production environment. No risk of accidentally charging a real card. - "Magic" customer names. If you/your test purchased as "Customer FailedBadCvv" then the payment will fail because the card CVV was bad. - Use test card numbers. Usually in combination with the magic customer names. - Know what specific scenarios you care about. If you don't sell into Colombia, don't bother writing a test for charging in COP. Architecturally, we had a few operating principles: - Never directly modify the database. We had internal endpoints (heavily restricted and audited) that allowed indirect edits (i.e., they passed through the API so we could do data input checks etc before writing), but that was the extent of it. - All webhook events are immediately stored. Quick 2xx response to the sender, then we handle processing (and retries) async as an internal concern. Senders do not retry infinitely, and when they retry it's often an exponential backoff over hours/days. Not the end of the world but not ideal. - Store amounts in the lowest minor unit of the currency. For USD, this means cents. - Build idempotency in from day 0. For payments, we attached a unique ID as metadata (the PK of the payment record) which came back in webhooks. Processor sends us two payment.succeeded webhooks with the same payment ID? Cool, process one, ignore the other. Overall: it's messy, it's complex, it's time consuming. You're not going to get it right the first, second, or third time. Don't try to boil the ocean -- focus on the smallest discrete area (unit tests) in the highest risk areas first, then iterate from there.

I would make the payment state machine the thing you test, not Stripe itself. Stripe test cards are useful, but they only cover the entry point. The scary cases are usually state transitions: paid in Stripe but not recorded locally, webhook arrives twice, webhook arrives late, provisioning succeeds but the email fails, user refreshes during a pending state. A practical workflow is to save a small set of real-looking webhook payload fixtures and replay them against a clean test database. Then add switches that force one step to fail: DB write, provisioning, email, UI polling. Each test should end by checking the final state users see, not just whether the webhook handler returned 200. That gives you repeatable failure drills instead of random manual chaos.