I am writing a bitcoin app and looking to implement a 'cancel' feature. All over reddit are references that if under 3 confirmation have occurred, technically a payment can be stopped. Maybe only a minute or 2 available, but still. I cannot find any reference in the bitcoin api docs to demonstrate how this would be done. I know that one altcoin uses an intentional 60 min gap to confirm for this very reason.
Anyone know how this is done?
Answer
Is it possible to cancel a Bitcoin transaction ...
... which was not yet broadcasted?
Yes, obviously it is possible. I'd propose you choose this method in your Bitcoin app by showing the user a confirmation screen for some seconds with information about the transaction and some buttons saying cancel and confirm. @nahtnam proposed a 60 minute delay which is too much, considering you may lose internet access or want the transaction to appear in the blockchain as soon as possible.
... which was already broadcasted?
Maybe... To cancel such a transaction, you'd need to create a block yourself which moves the inputs of the original transaction to one of your own addresses. (Effectively invalidating the original transaction.) However, creating a block costs several thousand dollars as of now and finding a block is not guaranteed.
Another possibility would be to broadcast another transaction taking the same inputs as the original one and targeting the outputs at one of your own addresses/wallets. To motivate miners to include this transaction instead of the original one, you increase the transaction fee. However, some clients may not relay such double spent transaction to the miners and some miners may reject the double spent transaction and include the original one (the one they received first), instead. (c.f. Bitpay encountered zero double spent in the first 10000 transactions. and Cancelling an unconfirmed transaction by @theymos (bitcoin.stackexchange))
There are some tricks to hide your initial transaction from miners by creating a so-called "non-standard" transaction. Alternatively, the transaction could include a very low fee to make miners reject it for economic reasons. (c.f. Significant losses by double-spending unconfirmed transactions (bitcoin-dev mailing list) and Double-spending by @petertodd (Reddit)) However, this makes your transaction look suspicious to the receiver if they look closer at it and they will most likely ask you to wait until it has one or more confirmations.
Finally, it is also possible if you set a flag on the original transaction to indicate replaceability. You can then replace the original transaction with another one by including the same (one or more) inputs in the replacement transaction. Also, you must pay a higher fee. However, not all miners honour this flag and some might still include your initial transaction. (c.f. Reference to BIP 125: Opt-in Full Replace-by-Fee Signaling)
... which is included in one or more blocks?
No, very unlikely to impossible. You'd need to control a substantial amount of hashing power to create a fork by rebuilding the blockchain starting at the block before the original transaction happened and ending at the block with height = (current public blockchain height) + 1. Therefore, "the more confirmations you have, the more difficult, expensive, and unreliable an attack like this is."
Source: @DannyHamilton (Bitcointalk)
The bitcoin paper by Satoshi Nakamoto explains that this is always possible when you control more than 50% of the hashing power, and possible with a probability less than 1, but greater than 0, if you control less than 50% hashing power. See Bitcoin: A Peer-to-Peer Electronic Cash System.
Though, if you control a substantial amount of hashing power, you are likely incentivised to not undermine the trust in bitcoin by undoing transactions and indirectly hurting your revenue stream from mining.