Pepperline charges dinner twice
A timed-out payment is retried without an idempotency key, so the card is charged again.
block payOrder · android 4.7.2
6 of 6 instances in 3.1 s. Menus and tracking keep working.
When an app release ships with a bug that costs money, your API refuses that release on the endpoints that cause harm.
For teams whose API serves iOS, Android and web apps
Small teams now ship app versions every week, and each release gets less human attention. Your backend can roll back in a minute. The app on a customer's phone cannot.
The companies are invented. The bugs are the kind that reach production every week.
A timed-out payment is retried without an idempotency key, so the card is charged again.
block payOrder · android 4.7.2
6 of 6 instances in 3.1 s. Menus and tracking keep working.
The app skips the seat hold, so two buyers complete the same seat in the same second.
block createOrder · ios 4.7.2
6 of 6 instances in 2.7 s. Browsing and web checkout keep working.
The app decides promo eligibility on its own and applies the first-ride discount to every trip.
block applyPromo · android 4.7.2
6 of 6 instances in 3.4 s. Riders still request, ride and pay.
Ads that fail to load are reported as watched, and each report grants premium gems.
block claimReward · android 4.7.2
6 of 6 instances in 2.8 s. Matches and purchases keep working.
A production build ships with test addresses, so withdrawals go where nobody holds the key.
block createWithdrawal · ios 4.7.2
6 of 6 instances in 2.9 s. Balances and deposits keep working.
Your team finds the bug. Upforcer blocks the release from the same thread, shows what the rule will match, and confirms when every instance is in sync. Nobody redeploys.
Maya Support · 18:55
Forty "charged twice" tickets in fifteen minutes. All from Android.
Tomás On-call · 18:57
Found it. 4.7.2 retries payOrder on timeout without an idempotency key. Blocking it until the fix ships.
/upforcer block payOrder android 4.7.2
upforcer APP 18:57
Draft rule r_42: block payOrder for Android 4.7.2 in prod. Preview: about 310 requests an hour match, all from Android 4.7.2. It is under the blast-radius limit, so no second approver is needed.
EnforceCancel
upforcer APP 18:58
Enforced by Tomás. All 6 instances in sync in 3.1 s. People on Android 4.7.2 now see "Update to 4.7.3 to order."
Maya Support · 19:04
Tickets stopped. Refunding the forty now.
Priya Payments · 21:30
4.7.3 is live with the idempotency key. Keep the block on. People will reopen 4.7.2 for months.
upforcer APP 21:30
Rule r_42 stays on. Anyone who opens 4.7.2 later, even months from now, gets the update screen instead of a double charge.
Each request names its release in a header. The middleware checks it against signed rules held in memory and answers with a 403 only for the blocked release and endpoint. A new rule reaches every instance in seconds.
One RFC 9651 structured header, set by the client SDK, with OpenTelemetry attribute names so the same values appear in your traces.
Client-Release: service.name="wallet",
service.version="4.7.2",
release.platform=ios,
release.build="1031",
deployment.environment.name=prod
The client SDK spots the refusal by the decision header and renders the update screen from the body. A served response from a version marked update-recommended carries Deprecation and Sunset headers instead.
HTTP/1.1 403 Forbidden Content-Type: application/problem+json Client-Release-Decision: action=update_required, rule="r_42", operation="createWithdrawal", min_version="4.7.3" { "type": "https://upforcer.dev/problems/update-required", "title": "Update required", "status": 403, "detail": "Withdrawals are paused for version 4.7.2. Update to 4.7.3 to continue.", "instance": "urn:upforcer:decision:01JBX7Q2M4", "operation": "createWithdrawal", "minimum_version": "4.7.3", "update_url": "https://apps.apple.com/app/id123" }
The editor shows which versions, endpoints and instances a draft matches and roughly how much of the last hour's traffic it would have blocked. A typo in a version range cannot lock out everyone. Dry run, an advanced mode, logs what a rule would block without blocking it.
draft block iOS 4.7.2 on createWithdrawal
matches 1 version · 1 endpoint · 6 of 6 prod instances
would block about 212 requests an hour
0 from iOS 4.7.1, iOS 4.6.9, Android 4.7.1
Each API instance keeps one outbound connection to the control plane. On enforce, the control plane signs the rule set and notifies the instances. Each one pulls it, verifies the signature against your project key and confirms. Nothing connects into your infrastructure.
rules v42, signed with k-main notified 6 instances 09:38:01 pulled 6 of 6, signature verified confirmed 6 of 6 09:38:04 (2.9 s)
Every request then names its release, and your dashboard lists the live versions before you write a single rule.
Sign in, name the project and copy its key into your API's environment.
UPFORCER_PROJECT=wallet UPFORCER_KEY=uf_sk_…
It reads the release header and decides in your process. Name the endpoints you may want to block.
// npm i @upforcer/hono
app.use(upforcer())
app.post("/orders/:id/pay",
upforcer.operation("payOrder"),
payOrder)
Also Express, Fastify, Next.js, NestJS, Go and Python.
It names the release on every request and shows the update screen on a refusal.
// Swift Package Manager
Upforcer.configure(
service: "wallet")
Also web and Kotlin.
That is the whole setup. Your live versions appear in the dashboard, and you block one when you need to.
Both are Apache 2.0 at github.com/upforcer. The server side decides, the app side announces itself and reacts.
Read the release header, decide in-process, refuse with a 403 and the decision header.
Send the release header on every request, and turn a refusal into an update screen or a typed error.
Plans are priced on what the control plane does for you, never on traffic.
Once a broken release is on your users' phones, every fix waits for app store review and for each user to update. The middleware goes into your API in minutes.
$ npm i @upforcer/hono
Also go get upforcer.dev/go, and SDKs for Python, web, Swift and Kotlin.
Upforcer is release quarantine for mobile and web apps. A broken release stays away from your money until the fix is in the store.