,

Credit Card Processing Down? Incident Response Runbook

·

Credit card processing down and checkout failing right now? Here’s the triage sequence to run in the first five minutes, manual failover options, and how to stop the next processor outage from taking down…

Credit card processing down means one of two things: your payment processor is having an outage, or something on your side is failing and looks like one. The first five minutes decide how fast you find out which, and every minute you spend guessing is a minute of failed checkouts.

This is a runbook, not a retail-store guide. If you’re looking for what to tell a cashier when a card reader loses WiFi, that’s not this article. This is for the engineer or on-call lead staring at a spike of failed transactions on a SaaS or platform checkout, trying to figure out whether to page the team, roll back a deploy, or just wait out someone else’s incident.

Credit Card Processing Down: Is It You or Your Processor?

Three signals tell you where the failure lives, and they take less time to check than it takes to open a support ticket.

Check the processor’s status page first. Every major PSP, including Stripe, Adyen, Braintree, and Worldpay, publishes a public status page or API status endpoint. If they’re reporting degraded service, you have your answer and your time is better spent on communication than debugging your own code.

Look at your error codes, not just your error rate. A spike in transaction failures with uniform errors (timeouts, connection resets, 5xx responses with no body) points to the processor. A spike with diverse errors (specific decline codes, validation failures, mixed success and failure on retries) points to something in your integration or the issuer‘s risk scoring, not a processor outage.

Correlate against your own deploy history. If the failure started within minutes of your last deploy, check that deploy before you check anyone else’s status page. That sounds obvious, but mid-incident it isn’t, because “the processor is down” is a more comfortable first assumption than “we shipped a bug.”

Total silence with no response at all is the strongest processor-side signal. A processor that’s up but rejecting transactions in specific, structured ways is usually telling you something about your request, not itself.

Immediate Triage Checklist

Once you’ve confirmed it’s the processor, work this sequence:

  1. Declare the incident internally. Page whoever owns payment infrastructure. Don’t wait to see if it resolves itself: a five-minute processor blip and a two-hour outage look identical in minute one.
  2. Check whether you have a second PSP connection already configured. If yes, this is a failover event, not a build event, so skip to the failover section below. If no, you’re in damage control until the processor recovers.
  3. Pull up your transaction volume dashboard and start a timestamped log. You’ll need this for reconciliation afterward and for the post-incident review. Note when failures started, what percentage of transactions are affected, and whether it’s total failure or partial degradation.
  4. Check for duplicate-charge risk before you retry anything automatically. If your checkout retries failed transactions client-side, confirm the processor isn’t accepting-then-timing-out, which can produce a successful charge that your system logged as failed.
  5. Start the customer and internal communication cadence (see below) before you have a root cause. Silence in minute five is worse than an honest “we’re investigating.”

Do not manually record and store a customer’s full card number to charge it later once the processor recovers. It’s a common improvisation during in-person outages and it violates PCI DSS card-data handling requirements as well as most processor merchant agreements. If you need to capture intent to pay, use a compliant deferred-authorization or tokenized retry mechanism, not a written-down number.

Common Failure Modes Behind a Processing Outage

Not every “processing is down” incident is the same failure. Orchestra’s breakdown of PSP error log patterns covers the full taxonomy of what generic decline codes actually mean and how to read them, and it’s worth bookmarking for the next incident rather than duplicating here. At a minimum, know which category you’re in.

Failure modeWhat it looks like
Full processor outageUniform timeouts or connection failures across all transaction types
Partial degradationLatency spikes and inconsistent success rates, not a hard failure
Network path failureProcessor is fine; DNS, a CDN, a load balancer, or another network hop between you and them is broken
Issuer-side decline spikeNot an outage; a card network or issuing bank is declining a higher-than-normal percentage of transactions, often risk-related

A network path failure looks identical to a processor outage from your dashboards until you test connectivity directly, and an issuer-side decline spike needs different handling than a true outage. Conflating the two wastes triage time.

The 2018 Visa Europe outage is worth knowing as a case study, because it shows how failure modes compound. A hardware fault in a primary data center switch should have triggered automatic failover to Visa’s secondary data center. It didn’t fail over cleanly: the fault prevented the backup switch from activating as designed, and engineers needed roughly ten hours and manual intervention to fully cut traffic over. Of 51.2 million transactions attempted during the window, 5.2 million failed (Finextra; root cause detail via Computer Weekly). Redundancy existed. The failover trigger was the single point of failure instead. That distinction matters for how you build your own failover, not just how you respond to this incident.

Manual Failover Options If You Have No Redundancy

If you don’t have a second PSP integration already live, your options during the incident itself are limited, and it’s worth being honest about that instead of pretending there’s a quick fix.

You cannot stand up a new processor integration mid-incident. Provisioning a merchant account, completing underwriting, and building the integration takes days at the absolute fastest, and typically weeks. Anyone telling you to “just switch processors” during a live P1 hasn’t built a payment integration before.

What you can do in the moment:

  • Queue transactions for retry once the processor recovers, with clear customer-facing messaging that the charge is pending, not failed, if your architecture supports safe idempotent retries.
  • Route new signups or non-time-sensitive transactions to a manual or delayed-billing flow if one exists, buying time without accumulating failed-transaction cleanup.
  • If you operate any transactions through a secondary rail for other reasons (a regional processor, a backup gateway used for testing), evaluate carefully, and only if pre-tested, whether it can absorb a subset of traffic. Untested failover paths introduce their own risk mid-incident.

Manual failover only works when the second connection already exists and the only thing you’re changing is a routing rule. That’s the entire argument for building redundancy before you need it rather than during.

Communicating the Outage to Customers, Support, and Leadership

Three audiences need different information at different cadences.

Customers need to know what’s affected and what to do, not a technical explanation. “Checkout is temporarily unavailable, we’re investigating” beats silence and beats over-explaining a root cause you haven’t confirmed yet. Update your status page and any in-app banners before support tickets pile up, not after.

Support and customer-facing teams need a script and a place to route questions, fast. Give them the same “checkout unavailable, working on it” language plus an internal channel for updates, so they’re not improvising answers that later turn out to be wrong.

Leadership needs impact numbers, not narrative, in the first update: percentage of transactions affected, estimated revenue impact if known, and your next update time. Then hit that update time. A missed self-imposed deadline does more damage to trust internally than the incident itself.

Why This Keeps Happening: The Single-Processor Trap

If this is not your first processing outage, the pattern is familiar: a single processor dependency means a single processor’s bad day becomes your bad day, in full, every time.

Key stat: Payment processing outages cost U.S. businesses $44.4 billion in lost sales annually (Payments Dive, 2024).

The damage compounds fast: businesses lose $1.2 billion per minute between minutes 8 and 13 of an outage, and 70% of vulnerable revenue is gone by minute 23 (Payments Dive, 2024). The average outage runs about two hours. Customers start abandoning checkout after roughly seven minutes. That gap between how long outages actually last and how long customers actually wait is a timing mismatch you can calculate against your own transaction volume, and it’s the entire business case for redundancy.

Engineering teams know this. The reason most companies still run single-processor is the cost of building the alternative, not disagreement about the risk.

How Orchestration Prevents the Next Outage

A payment orchestration layer sits between your checkout and multiple PSPs, so a single processor outage becomes a routing decision instead of a full checkout outage. When your primary processor starts failing, whether by error rate, latency, or a hard status-page confirmation, traffic reroutes to a healthy backup processor automatically, without a human deciding to flip a switch mid-incident.

That’s the practical difference between this article’s triage checklist and not needing this article at all: automatic failover turns “declare incident, page the team, manually assess options” into a rerouted transaction that a monitoring dashboard logs and nobody pages anyone about. Orchestra’s payment gateway failover is built for exactly this, detecting processor-side failure and rerouting before it becomes a checkout-wide incident, using the same multi-PSP connections you’d otherwise be racing to build during a live outage.

This doesn’t eliminate the need for the triage skills above. Network path failures, issuer-side decline spikes, and your own deploy-correlated bugs still need the same diagnostic sequence. What changes is that a processor-side outage, the failure mode that takes down 100% of checkout on a single-processor stack, stops being the worst-case scenario.

Post-Incident: Building a Failover Runbook for Next Time

Once the immediate incident is resolved, three things turn this into a lesson instead of a repeat:

Write the timeline while it’s fresh. When did failures start, when were they confirmed as processor-side, when did service resume, what was the transaction and revenue impact. This becomes your evidence for prioritizing redundancy work.

Reconcile transactions carefully. Retried and queued transactions during an outage can produce duplicate charges or missed charges. Match every retry against the original attempt before closing the incident.

Turn the timeline into the redundancy conversation. Multi-processor redundancy improves authorization rates by 3-5% on top of eliminating single-processor outage risk entirely, and it’s rarely a hard sell once a team has just lived through an actual outage. The harder question is build versus buy: building failover in-house typically takes engineering teams 3-6 months; an orchestration layer with failover already built can cut that to 2-3 weeks, because the routing and normalization work is already done.

If you want the deeper architecture read, covering active-active versus active-passive failover, SLA specifics, and the full build-vs-buy TCO breakdown, Orchestra’s high availability architecture guide covers that ground. This runbook is deliberately narrower: what to do while it’s happening, not how to architect around it happening again.

Frequently Asked Questions

My credit card processing is down right now. What do I do first?

Check your PSP’s public status page, then check your own error logs for a spike in a specific error code versus total silence. Total silence with no response points to the processor; a spike in a specific decline code points to your own integration or the issuer.

How do I tell if it’s my payment processor or my own system that’s down?

Check three signals in order: the processor’s status page, whether errors are diverse (your side) or uniform timeouts (their side), and whether the failure correlates with your last deploy. A deploy correlation means check your own change before anything else.

Can I manually switch to a backup payment processor during an outage?

Only if a second processor integration already existed before the incident. Standing up a new PSP connection mid-outage takes days to weeks at minimum; manual failover is only fast when the second connection already exists and routing can flip at the config level.

What’s the difference between a processor outage and a spike in declined transactions?

An outage is infrastructure-level: timeouts, 5xx errors, or no response, affecting all transaction types uniformly. A decline spike is usually issuer-side risk scoring or a specific decline code concentrated in one segment, and it needs different handling than a true outage.

How should I communicate a payment outage to customers?

State what’s affected (checkout, not the whole site), avoid speculating on cause before you’ve confirmed it, and give a next-update time you can actually hit. Missing your own stated update window damages trust more than the outage itself.

How long do payment processor outages typically last?

About two hours on average, though duration varies widely by cause. Customers typically abandon checkout after about seven minutes of failure, meaning most outages far outlast customer patience (Payments Dive, 2024).

How do I stop this from happening again?

Add a second PSP connection with automatic failover so a single processor outage no longer means a full checkout outage. Better alerting tells you faster that revenue stopped, but it doesn’t stop it from stopping; only redundant routing changes the outcome.

More recent articles