Blog  waxwing's bloghttps://joinmarket.me:8001/blog/blog/20190428T14:36:26+00:00BlogMultiparty S620190428T14:36:26+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/multipartys6/<h2>The multiparty symmetrical Schnorr signature scriptless script shuffle</h2>
<p>This blog is in the category of "a newish idea about privacy tech"; like similar previous ones (e.g.: <a href="https://joinmarket.me/blog/blog/coinjoinxt/">CoinJoinXT</a>) it is little more than an idea, in this case I believe it is correct, but (a) I could be wrong and there could be a flaw in the thinking and (b) it's not entirely clear how practically realistic it will be. What I do hope, however, is that the kernel of this idea is useful, perhaps in Layer 2 tech or in something I haven't even thought about.</p>
<p></p>
<h2>The Goal</h2>
<p>As with similar writeups, I feel it's important that the reader has some idea what the goal is. Here is the goal I <em>mostly</em> had in mind when thinking this through:</p>
<p></p>
<ul>
<li>11 (or 6, or 24...) anonymous users coordinate (on a lobby server, with a Joinmarketstyle incentive, on a p2p network somehow  whatever). They each have 1 BTC utxos (put off denomination questions for later) and they want a very meaningful privacy increase.</li>
<li>Instead of doing a CoinJoin which is obvious or a whole set of CoinSwaps (see <a href="https://joinmarket.me/blog/blog/coinswaps/">earlier</a> <a href="https://joinmarket.me/blog/blog/flippingthescriptlessscriptonschnorr/">posts</a>) which could get complicated for 11 people, they want to kind of "permute" or "shuffle" all their utxos.</li>
<li>It's a year or two from now and a Schnorr soft fork has gone through in Bitcoin mainchain; they're going to use the scriptless script primitive (see <a href="https://joinmarket.me/blog/blog/flippingthescriptlessscriptonschnorr/">here</a> or Poelstra and Nick's writeup <a href="https://github.com/apoelstra/scriptlessscripts/blob/master/md/atomicswap.md">here</a>, or the following sections for more on this), to achieve the goal via multisig outputs that look like other outputs.</li>
<li>They do effectively a "multiparty swap" or "shuffle" to achieve this goal. Each of the 11 participants funds a single prepared destination address, which is (though not seen because Schnorr) an 11 of 11 multisig. Before they do so, they get hold of a presigned (by everyone) backout transaction to get their coins back if something goes wrong.</li>
<li>They decide a shuffle/permutation: e.g. Alice is paying Bob 1, Charlie is paying Edward etc etc. ... we're talking here about a member of the set of permutations of 11 objects. Obviously the idea is that everyone pays in 1, everyone gets back 1. They prepare transactions for these payments.</li>
<li>Once everything is set up they pass around <strong>adaptor signatures</strong> which create the atomicity effect we want  when any 1 of the 11 transactions goes through, all of them will go through.</li>
<li>In a certain order, that we'll discuss, they can now pass real (Schnorr) signatures (note that even though "real" they are still "partial" signatures  they're 1 of 11 needed in the multisig) on the transactions such that one member of the group has a full set and can broadcast the transaction paying themselves. Everyone else sees this on the blockchain, and combining the signatures in this published transaction, with the earlier adaptor signatures, has enough information to broadcast the other transaction which pays themself.</li>
</ul>
<p>Let's consider the advantages of doing this:</p>
<ul>
<li>Shared with a 2 of 2 CoinSwap: there is no linkage on the blockchain between the 11 transactions. Effectively, Alice has swapped her coin history with Bob, Charlie with Edward etc..</li>
<li>Big difference from the above: we can create, like a multiparty coinjoin, the highly desirable scenario that <span style="textdecoration: underline;">individual participants do not know the linkages</span> between inputs for transactions other than their own. As we know, there are various designs of CoinJoin metaprotocols that allow this to different extents, but if CoinSwap is restricted to 2 of 2 this is impossible (no cryptographic trickery prevents the deduction "if it's not mine, it's yours!").</li>
<li>Biggest difference from CoinJoin is that CoinSwap transactions (whether 22 or 1111) can look like ordinary payments on the blockchain, although there's meaningful wiggle room in how exactly they will look. If we manage to combine this with even slight variations in size of individual payments, and probably a little timing decorrelation too, the task of a blockchain analyst in identifying these is near impossible (<strong>notice that this hugely desirable steganographic feature is shared with PayJoin and CoinJoinXT, previous blog posts</strong>  notice though, that it depends on Schnorr for indistinguishable multisig <em>and </em>for adaptor signatures, unless ECDSAnparty computation is a thing, which I doubt is currently a thing for more than 2 parties, but see e.g. <a href="https://eprint.iacr.org/2018/987.pdf">this</a> for recent research in this area.).</li>
</ul>
<p></p>
<h3>Illustrations comparing CoinJoin, CoinSwap and multiparty S6:</h3>
<p><img alt="Typical coinjoin" height="436" src="https://joinmarket.me/static/media/uploads/.thumbnails/screenshot_from_20190118_150033.png/screenshot_from_20190118_150033813x436.png" width="813"/></p>
<p><em>Typical CoinJoin transaction  it's very obvious because of equal output amounts; the histories of coins are not disconnected, but fused</em></p>
<p><em></em></p>
<p><em><img alt="" height="65" src="https://joinmarket.me/static/media/uploads/.thumbnails/realcoinswap2.png/realcoinswap2792x65.png" width="792"/></em></p>
<p><em><img alt="" height="67" src="https://joinmarket.me/static/media/uploads/.thumbnails/realcoinswap1.png/realcoinswap1799x67.png" width="799"/></em></p>
<p><em>Typical 2party CoinSwap transactions; they are entirely separate on the blockchain, with different timestamps they could be extremely difficult to find.</em></p>
<p><em></em></p>
<p><em><img alt="" height="745" src="https://joinmarket.me/static/media/uploads/.thumbnails/s6basic.png/s6basic1053x745.png" width="1053"/><br/></em></p>
<p><em>A <strong>very </strong>simplified multiparty S6 as envisaged: note that Oscar to Peter shows on a diagonal a simple transaction of the type used in CoinSwap; in fact there is one such transaction for every red arrow; i.e. each red arrow represents a payment from one of the group of 11 to another, in a random permutation. All of these transactions will be atomic; either they will all happen or none will. But none will be linked on the blockchain.</em></p>
<h2>Schnorr and adaptor signatures</h2>
<p>Achieving the goals above is crucially dependent on the concept of an adaptor signature as developed by Andrew Poelstra (see some detailed descriptions as mentioned <a href="https://github.com/apoelstra/scriptlessscripts/blob/master/md/atomicswap.md">here</a>) in his work on "scriptless scripts". A large part of the <a href="https://joinmarket.me/blog/blog/flippingthescriptlessscriptonschnorr/">earlier blog post</a> on the topic of the scriptless script based swap, was explaining this concept. I want to write an explanation which is easier to understand. I will try :)</p>
<p>A basic Schnorr signature on a message \(m\) using a public key \(P\) whose private key is \(x\), looks like this:</p>
<p>\(\sigma = k + \mathbb{H}(PRm) x \quad, R = kG \quad \textrm{Publish: }\ (R,\sigma)\)</p>
<p>\(k\) is called the nonce, and \(R\) is the nonce point (point on the curve corresponding). We shorten the hash function \(\mathbb{H}(\ldots)\) to just \(e\), often.</p>
<p>Schnorr signatures are linear in the keys, in that:</p>
<p>\(\sigma_1 + \sigma_2 = (k_1 + k_2) + e (x_1+x_2)\)</p>
<p><strong>Combining signatures in this way is unsafe in many contexts, in particular multisignature in Bitcoin. See the <a href="https://eprint.iacr.org/2018/068">paper on Musig</a> and <a href="https://github.com/ElementsProject/secp256k1zkp/blob/secp256k1zkp/src/modules/musig/musig.md">this summary</a> for the details on how the weakness (basically, potential of key subtraction) is addressed in detail, using interactivity between the parties cooperating to create the agreggated Schnorr signature. <span style="textdecoration: underline;">As long as this is properly addressed, though, the linearity property is retained</span>.<br/></strong></p>
<p><strong>Let me emphasise that the rest of this post will ignore the correct construction of keys and nonce points for safe Schnorr multisig; we will just talk about Alice, Bob and Charlie adding keys together and adding signatures together; the difference is crucial in practice but I believe does not alter any of the concepts being outlined.</strong></p>
<h3>Partial Signatures</h3>
<p>With the above bolded caveats in mind, it'll be important for the following to understand the idea of a "partial signature" in a Schnorr multisig context. What we're doing is to create a single signature \(\sigma\) which represents, say, a 2 of 2 multisignature. Say it's Alice and Bob (A, B). Then Alice would produce this <strong>partial signature</strong>:</p>
<p>\(\sigma_A = k_A + \mathbb{H}(P_A + P_B  R_A + R_B  m) x_A\)</p>
<p>Notice how it's not a valid signature according to the Schnorr definition because the nonce \(k_A\) does not correspond to the nonce point \(R_A + R_B\) <em>and </em>because the private key does not correspond to the public key \(P_A+P_B\).</p>
<p>However when Bob adds his partial signature:</p>
<p>\(\sigma_B = k_B + \mathbb{H}(P_A + P_B  R_A + R_B  m) x_B\)</p>
<p>... to Alice's, the sum of the two <em>is </em> a valid signature on the message, with the sum of the keys.</p>
<p>We will make use of this shortly.</p>
<h3>Adaptor Signatures</h3>
<p>A creator of a signature can hide a verifiable secret value in a signature, using simple addition. They can then pass across the signature <em>without </em>the secret value, making it not valid, but verifiable as "a signature with the secret not included". This is what Poelstra means by his concept "adaptor signature". It looks like this:</p>
<p>\(\sigma' = k + \mathbb{H}(PR+Tm) x \quad R=kG,\ T=tG \)</p>
<p>(from now on, note that ' indicates adaptor signatures). To repeat, it's not a valid signature, but: <em>it can be verified that adding the discrete log of \(T\)</em><em> to \(\sigma'\) will yield a valid signature on the message m and the public key \(P\)</em><strong><em>.</em></strong><em></em></p>
<p>Refer back to the <a href="https://joinmarket.me/blog/blog/flippingthescriptlessscriptonschnorr/">earlier blog post</a> if you want to check the mathematical details on that.</p>
<p>The alert reader will notice how similar the "adaptor signature" and "partial signature" concepts are  it's almost the same mathematical change, but with a very different purpose/application, as we expand on below:</p>
<h3>Atomicity for two parties</h3>
<p>This trick is already cool  if I ever pass you the secret value \(t\), you'll be able to form a full valid signature. But with an additional nuance it's possible to make this a twoway promise, so we have the outline of an atomicity property, which can be very powerful. If the contents of the thing being signed, including the nonce points \(R\) and the public keys \(P\) are fixed in advance, we could create a situation where the promise works both ways:</p>
<p>\(P, R, m\) are known \(\therefore \sigma = k + t + \mathbb{H}(PR+Tm) x\) is fixed. If the adaptor is shared by the owner of the private key \(x\), i.e. if he passes to a counterparty the value \(\sigma' = k + \mathbb{H}(PR+Tm) x\), then either direction of publication reveals the other:</p>
<ul>
<li>If the full signature \(\sigma\) is revealed, the secret is revealed as \(t = \sigma  \sigma'\)</li>
<li>If the secret \(t\) is revealed, the full signature is revealed: \(\sigma = t + \sigma'\)</li>
</ul>
<p>This atomicity is the basic of the scriptless script atomic swap published by Poelstra and explained in my earlier post.</p>
<h3>Atomicity for N parties</h3>
<p><strong>This is the novel idea in this post.</strong></p>
<p>Suppose we have fixed \(\Sigma P\), \(\Sigma R\) and a single message \(m\). In other words several participants signing together the same message (Bitcoin transaction).This is the scenario for Schnorr aggregated multisig, modulo the complexity of Musig which as explained above I'm deliberately ignoring. Without adaptors, each party will have to produce a <strong>partial signature<em> </em></strong><em></em>as already described, and then they can all be added together to create a fully valid Schnorr signature.</p>
<p>Now suppose each of 3 parties (Alice, Bob, Charlie) makes an adaptor signature for their partial signature:</p>
<p>\(\sigma_A' = k_A + \mathbb{H}(\Sigma P  \Sigma R + \Sigma T  m) x_A\)</p>
<p>Little explanatory note on this: each party will have to share their public \(T\) values (which, remember are curve points corresponding to the <em>adaptor secrets</em> \(t\)), so they will all know how to correctly calculate the hash preimage by "combining" (here just adding, but with musig it's more complicated) their public keys, and then linearly adding in all their \(T\) public values to the corresponding \(R\) nonce points as for a normal Schnorr signature.</p>
<p>Similarly for Bob, Charlie:</p>
<p>\(\sigma_B' = k_B + \mathbb{H}(\Sigma P  \Sigma R + \Sigma T  m) x_B\)</p>
<p>\(\sigma_C' = k_C + \mathbb{H}(\Sigma P  \Sigma R + \Sigma T  m) x_C\)</p>
<p>These can then be shared (all with all) and are verifiable in the same way as previously, e.g.:</p>
<p>\(\sigma_A' G \stackrel{?}{=} R_A + \mathbb{H}(\Sigma P  \Sigma R + \Sigma T m) P_A \)</p>
<p>But, it seems to get a bit confusing when you ask what happens if one party reveals either a full \(\sigma\) value, or a secret \(t\).</p>
<p>For example, what if Alice reveals her full partial signature (yes I meant that!) \(\sigma_A =k_A + t_A + \mathbb{H}(...) x_A\)?</p>
<p>One partial signature on its own is not enough for Bob or Charlie to do anything. If Alice <em>and </em>Bob do this, and pass these partials to Charlie, then he can complete and publish. But we want atomicity. What we want is:</p>
<ul>
<li>If the complete transaction signature is ever published, <em></em>all parties can learn the adaptor secrets \(t\).</li>
<li>If any or all parties learn the adaptor secrets \(t\) they can publish the complete transaction.</li>
</ul>
<p>It's clear why that's the desire: that would mean you could make multiple different transactions, sharing the same set of adaptor secrets, and have it so that <span style="textdecoration: underline;">if one transaction gets published all the others do!</span></p>
<p><strong>But wait!</strong> Something was horribly obfuscated in those bullet points. "learn the adaptor secrets"? All of them, or which ones?<strong><span style="textdecoration: underline;"></span></strong></p>
<p>That this is crucial is easily seen by considering the following "attack":</p>
<p>Suppose Alice, Bob, Charlie make three transactions, each of which pays out of a 3of3 Schnorr multisig, between them. The idea would be (as you've probably gathered from the buildup) that if any 1 transaction, say the first one paying Bob, gets broadcast, then both Alice and Charlie could broadcast the other 2 transactions, paying each of them, because they "learnt the adaptor secrets". But: if say Alice kicks off, reveals her adaptor secret \(t_A\), then couldn't Bob and Charlie collude? They could take the partial signature of Alice:</p>
<p>\(\sigma_A = k_A + t_A + \mathbb{H}(\ldots)x_A\)</p>
<p>and then between themselves share and construct their "joint" partial signature:</p>
<p>\(\sigma_{BC} = k_B + k_C + t_B + t_C +\mathbb{H}(\ldots)(x_B+x_C)\)</p>
<p>then add this to \(\sigma_A\). They could do this for the two transactions paying <em>them</em> and publish them to the blockchain. It may seem at first glance that this is a problem, because in doing so they haven't revealed their <em>individual</em> adaptor secrets \(t_B, t_C\) but have instead revealed their sum \(t_B+t_C\).</p>
<p>However this is not a problem! One way of looking at it is <strong>adaptor signatures are just as linear as proper Schnorr signatures</strong>. They are thus aggregatable. From Alice's point of view, although she is taking part in a 3of3 Schnorr multisig, she may just as well be participating in a 2of2 with a single party, if Bob and Charlie choose to collude and combine in that way. What will Alice see on the blockchain?</p>
<p>\(\sigma = k_A + k_B + k_C + t_A + t_B + t_C + \mathbb{H}(\ldots)(x_A+x_B+x_C)\)</p>
<p>But she already got adaptor signatures from Bob and Charlie, so she can remove them:</p>
<p>\(\sigma  \sigma_A  \sigma_B'  \sigma_C' = t_B + t_C\).</p>
<p>Now possessing the value of the <em>sum</em> \(t_B+t_C\), she can add this to preexisting adaptor signatures for the transaction paying <em>her </em>and get the complete multisignature on those!</p>
<p>Unfairly linear signatures for the win!</p>
<h2>Protocol Outline</h2>
<p>We've now covered the meat of the concepts; so this instantiation phase will either be easy to follow, or, if you found the above slightly opaque, will hopefully make it clearer by being more concrete.</p>
<p>We're now ready to outline this "multiparty SSSSSS" design :) We'll break it into three phases.</p>
<p>Because 11 party would be too tedious, we'll stick to Alice, Bob, Charlie three party case as above.</p>
<p>Getting clear on notation: \(D_x\) will be destination addresses, transactions will be \(\tau_x\), adaptor secrets will be \(t_x\), corresponding curve points: \(T_x\). Signatures will be \(\sigma_x\) and adaptor signatures will be marked with a ', so \(\sigma_{x}'\). The subscripts will almost always be one of \(A, B, C\) for Alice, Bob, Charlie.</p>
<h2>Phase 1  Setup</h2>
<p>We first negotiate three destination addresses: \(D_A, D_B, D_C\). Here the subscripts denote the payer <strong>into</strong> the address. So after the end of the setup the first will contain a coin paid by Alice, the second by Bob and the third by Charlie. The preparation of these addresses/keys will of course be done with Musig but to reiterate, we are ignoring the complexity there.</p>
<p>The three parties then all provide signatures on backout transactions such that each party gets their money back after a timeout. See the section "Timing Controls" for more details on this.</p>
<p>Once backouts are presigned, all parties pay into the destinations as above and wait for confirms.</p>
<p>Parties will agree in advance on the "shuffle"/permutation of coins, i.e. who will be paying who; this is related to Timing Control, so again, see that section. The exact negotiation protocol to decide the permutation is left open here. Once agreed, we know that we are going to be arranging three transactions paying out of \(D_A, D_B, D_C\), we'll call these \(\tau_{AB}, \tau_{BC}, \tau_{CA}\) respectively, where the second subscript indicates who receives the coin.</p>
<p></p>
<h2>Phase 2  Adaptors</h2>
<p>Each participant chooses randomly their adaptor secret \(t_A, t_B, t_C\) and then shares \(T_A, T_B, T_C\) with all others (<em>technical note: this might need to happen in setup phase</em>). They then also <strong>all</strong><em> </em>provide adaptor signatures on <strong>all</strong> of three transactions \(\tau_{AB}, \tau_{BC}, \tau_{CA}\) to each other. Note that there is no risk in doing so; the adaptor signatures are useless without receiving the adaptor secrets. Now each party must make two checks on each received adaptor signature:</p>
<ul>
<li>That it correctly matches the intended transaction \(\tau\) and the set of agreed keys in the setup</li>
<li>Crucially that <em>each </em>of the adaptor signatures from any counterparty correctly matches the same adaptor secret point \(T\).</li>
</ul>
<p>("Crucial" of course because without using the same adaptor secrets, we don't get our desired atomicity across a set of transactions).</p>
<p>To be more concrete, here are the actions of Bob:</p>
<ol>
<li>Generate a single adaptor secret randomly: \(t_B \stackrel{$}{\leftarrow} \mathbb{Z}_N\)</li>
<li>Broadcast \(T_B = t_B G\) to Alice, Charlie</li>
<li>Having agreed on all three payout transactions \(\tau_{AB}, \tau_{BC}, \tau_{CA}\), generate three adaptor signatures: \(\sigma_{B\tau_{AB}}' = k_{B\tau_{AB}} + \mathbb{H}(\Sigma P_{AB}  \Sigma R_{AB} + \Sigma T  m_{\tau_{AB}}) x_{B\tau_{AB}}\), \(\sigma_{B\tau_{BC}}' = k_{B\tau_{BC}} + \mathbb{H}(\Sigma P_{BC}  \Sigma R_{BC} + \Sigma T  m_{\tau_{BC}}) x_{B\tau_{BC}}\), \(\sigma_{B\tau_{CA}}' = k_{B\tau_{CA}} + \mathbb{H}(\Sigma P_{CA}  \Sigma R_{CA} + \Sigma T  m_{\tau_{CA}}) x_{B\tau_{CA}}\).</li>
<li>Broadcast these adaptors to Alice, Charlie.</li>
<li>Receive the 2 x 3 = 6 corresponding adaptors from Alice and Charlie. Verify each one (note the above bullet points).</li>
</ol>
<p>Assuming all parties accept the adaptor signatures, we are ready to proceed to the last phase. If any communication or protocol failure occurs, all parties must fall back to the backout transactions presigned in the Setup phase.</p>
<p>(<em>Technical note: it is not necessary for all parties to share all adaptors, in general, but for simplicity we use that model, since it hurts nothing I believe)</em>.</p>
<h2>Phase 3  Execution</h2>
<p>The order of events in this final execution phase is important for safety, but we defer that to the next section "Timing Controls". Here we'll just show how events will proceed if everything goes correctly, in a randomly chosen order.</p>
<ul>
<li>Alice and Charlie send full partial(! i.e. not adaptor) signatures on \(\tau_{AB}\) to Bob, i.e. the transaction that pays Bob. So they send \(\sigma_{A\tau_{AB}}\) and \(\sigma_{C\tau_{AB}}\), respectively.</li>
<li>Bob can add this to his own full partial signature on the same transaction, constructing: \(\sigma_{\tau_{AB}}\) and using this to broadcast \(\tau_{AB}\) to the network, receiving his coin.</li>
<li>Alice will read \(\sigma_{C\tau_{AB}} + \sigma_{B\tau_{AB}} = \sigma_{\tau_{AB}}  \sigma_{A\tau_{AB}}\) from this broadcast signature and from this deduce the value of \(t_B+t_C = \sigma_{C\tau_{AB}} + \sigma_{B\tau_{AB}}  \sigma_{C\tau_{AB}}' + \sigma_{B\tau_{AB}}'\).</li>
<li>Alice can add this aggregated adaptor secret \(t_B+t_C\) to the preexisting adaptors \(\sigma_{B\tau_{CA}}' + \sigma_{C\tau_{CA}}'\) to get \(\sigma_{B\tau_{CA}} + \sigma_{C\tau_{CA}}\), which she can then add to \(\sigma_{A\tau_{CA}}\) to get a fully valid \(\sigma_{\tau_{CA}}\) and broadcast this to the network to receive her 1 coin.</li>
<li>Charlie can do exactly the same as Alice for the last transaction, \(\tau_{BC}\) and receive his coin.</li>
</ul>
<p>Thus both other parties, after the first spend, were able to claim their coin by creating complete signatures through combining the adaptor signatures with the revealed (possibly aggregated) adaptor secrets. (<em>Technical note: in a protocol we can allow participants to share adaptor secrets at the appropriate times instead of having it deduced from transaction broadcasts, as in the case of CoinSwap, just as a kind of politeness, but this is not important</em>).</p>
<p></p>
<h2>Timing Controls</h2>
<p>In the 2 party scriptless script swap, as in earlier CoinSwap designs, we simply account for the asymmetry of timing of revealing priviliged information (e.g. signatures) using an asymmetry of timelocks. The one who transfers something valuable first (a signature) must have an earlier ability to refunds coins that are "stuck" due to protocol noncompletion, else the possessor of the adaptor secret / CoinSwap secret, who does not reveal it first, may wait for the window where he can reclaim and the other cannot, to both reclaim and use the secret to steal the other's coins.</p>
<p>Here we must follow a similar principle, just extended to multiple parties.</p>
<p>Suppose, naively, we just used the same locktime on each of the three refund transactions.</p>
<p>Now suppose Alice, at the start of Phase 3, reveals her full signature first, on transaction \(\tau_{AB}\) which pays Bob. And suppose for maximal pessimism that Bob and Charlie are colluding to defraud Alice. They will simply wait until the moment of the timeout and attempt to cheat Alice: try to broadcast both of their own refunds, while spending the transaction for which Alice provided the full signature (having done so, she has revealed her adaptor secret to the other two).</p>
<p>Thus by instead making Alice's backout locktime the earliest, she is safe in transferring her full signature, and thus her adaptor secret first. In this case if Bob and Charlie collude, they can do no better than publish this spend before that (earliest) timeout, and in so doing, reveal the aggregate of their adaptor secrets atomically so Alice can claim her money well before their backouts become active, as intended by system design.</p>
<p>Now let's consider the second sender of a full signature, say it's Bob. Suppose we let Charlie's locktime be identical to Bob's. And for maximal pessimism let's say Alice and Charlie collude. Here, Charlie could refuse to pass his signature to Bob and attempt to reclaim his coin at the exact moment of the timeout, while spending Bob's (depending on the exact permutation of spends, but at least, it's possible). Even though Alice didn't back out at her timeout in this scenario, which is weird, clearly this scenario is not safe for Bob, he has passed across a signature to Charlie with no time based defence against him.</p>
<p>These considerations make it obvious, I think, that the obviously sound way to do it is to stagger the locktime values according to the order in which signatures, and therefore secrets, are revealed. If the order of signature transfers is: first Alice, then Bob, then Charlie, then the locktimes on the backouts which pay each must obey \(L_A < L_B < L_C\).</p>
<p>So I believe this ordering must be settled on in Phase 1 (because we define these locktimes before signing the backout transactions).</p>
<p></p>
<h2>Generalisation from 33 to NN.</h2>
<p>I believe this is trivial, modulo practicality.</p>
<p></p>
<h2>Practical considerations, advantages</h2>
<p>The scenario described is a multiparty coinswap in which essentially a group of \(N\) parties could randomly shuffle history of their coins. This could be done with or without a coordinator (either Joinmarket style or server style), could possibly be done with a Coinshuffle++ type coordination mechanism and/or blinding.</p>
<p>Practicality: the biggest limitation is that of CoinSwap generally, but extended further: using staggered locktime backouts means that in cases of failure, participants may have to wait a long time for coin recovery. This gets linearly worse with anonymity set, which is not good. Would love to find a trick to avoid that.</p>
<p>Expanding further on that same limitation, a larger number of participants makes worse the problem that I've previously called "XBI", or "cross block interactivity". There's a lot of exposure to DOS attacks and simple network failure when participants not only have to coordinate, but have to do so more than once. This could partially be addressed with incentives, e.g. fidelity bonds, but I'm not convinced.</p>
<p>On the positive side, there could be a tremendous boon over the 2party case in that it's possible here to have a group of anonymous participants shuffle the history of their coins without any of the parties knowing the others' linkages.</p>
<p>Also positively, such larger group swaps may offer much larger privacy improvements in a very economical way (a few hundred bytes on chain per participant vs tens or hundreds of kilobytes via coinjoin? complete finger in the air here of course).</p>
<p>Leaving open: can amount decorrelation be achieved in a more powerful way in this model? I believe so, for example by splitting amounts into subsets across subsets of participants in interesting ways. Fees can also be used for noise. I think the most powerful version of this model would be very powerful indeed, but needs more detailed analysis, and this blog post is already too long.</p>
<p>Other applications: also leaving this open. Perhaps using adaptor signatures in groups like this (exploiting the linearity of adaptor signatures) has applications to second layer tech like Lightning or similar contracting.</p>Ring signatures20190220T23:06:47+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/ringsignatures/<h2>Outline:</h2>
<ul>
<li>Basic goal of 1of\(N\) ring signatures</li>
<li>Recap: the \(\Sigma\)protocol</li>
<li>OR of \(\Sigma\)protocols, CDS 1994</li>
<li>AbeOhkuboSuzuki (AOS) 2002 (broken version)</li>
<li>Security weaknesses</li>
<li>Key prefixing</li>
<li>Borromean, MaxwellPoelstra 2015</li>
<li>Linkability and exculpability</li>
<li>AND of \(\Sigma\)protocols, DLEQ</li>
<li>LiuWeiWong 2004</li>
<li>Security arguments for the LWW LSAG</li>
<li>Back 2015; compression, singleuse</li>
<li>FujisakiSuzuki 2007 and Cryptonote 2014</li>
<li>Monero MLSAG</li>
</ul>
<h2>Basic goal of 1of\(N\) ring signatures</h2>
<p>The idea of a <a href="https://en.wikipedia.org/wiki/Ring_signature">ring signature</a> (the term itself is a bit sloppy in context, but let's stick with it for now) is simple enough:</p>
<p>An owner of a particular private key \(x\) signs a message \(m\) by taking, usually without setup or interaction, a whole set of public keys, one of which is his (\(P=xG\)), and forms a signature (exact form unspecified) such that there is proof that <strong>at least one</strong> of the private keys is known to the signer, but which one was responsible for the signature is not known by the verifier, and not calculatable.</p>
<p>Obviously that's pretty vague but captures the central idea. We often use the term "ring" because the construction must have some symmetry over the entire set of \(n\) public keys, and a ring/circle represents symmetry of an arbitrarily high order (limit of an \(n\)gon). Less abstractly it could be a good name because of some "loop"ing aspect of the algorithm that constructs the signature, as we'll see.</p>
<p>What properties do we want then, in summation?</p>
<ul>
<li>Unforgeability</li>
<li>Signer ambiguity</li>
</ul>
<p>We may want additional properties for some ring signatures, as we'll see.</p>
<p>In the following sections I want to cover some of the key conceptual steps to the kinds of ring signatures currently used in cryptocurrency protocols; most notably Monero, but also several others; and also in the Confidential Transactions construction (see: Borromean ring signatures, briefly discussed here). I will also discuss security of such constructions, in much less detail than the <a href="https://joinmarket.me/blog/blog/liarscheatsscammersandtheschnorrsignature/">previous blog</a> (on the security of Schnorr signatures), but showing how there are several tricky issues to be dealt with, here.</p>
<h2>Recap: the \(\Sigma\)protocol</h2>
<p>We consider a prover \(\mathbb{P}\) and a verifier \(\mathbb{V}\).</p>
<p>A \(\Sigma\)protocol is a three step game, in which the prover convinces the verifier of something (it can be \(\mathbb{P}\)'s knowledge of a secret, but it can also be something more complicated), in zero knowledge. Readers interested in a much more detailed discussion of the logic behind this and several applications of the idea can read Sections 3 and 4 of my <a href="https://github.com/AdamISZ/from0k2bp">From Zero (Knowledge) to Bulletproofs</a> writeup, especially section 4.1.</p>
<p>In brief, the three step game is:</p>
<p>\(\mathbb{P} \rightarrow \mathbb{V}\): <strong>commitment</strong></p>
<p>\(\mathbb{V} \rightarrow \mathbb{P}\): <strong>challenge</strong></p>
<p>\(\mathbb{P} \rightarrow \mathbb{V}\): <strong>response </strong></p>
<p>A few minor notes on this: obviously the game is not literally over with the response step; the verifier will examine the response to establish whether it is valid or invalid.</p>
<p>The <strong>commitment</strong> will usually in this document be written \(R\) and will here always be a point on an elliptic curve, which the prover may (or may not! in these protocols) know the corresponding scalar multiple (private key or nonce) \(k\) such that \(R=kG\).</p>
<p>The <strong>challenge</strong> will usually be written \(e\) and will usually be formed as the hash of some transcript of data; the subtleties around exactly <em>what</em> is hashed can be vitally important, as we'll see. (This is in the "FiatShamir transform" case; we discussed the pure interactive challenge case a bit in the previous blog and many other places!)</p>
<p>The <strong>response</strong> will usually be a single scalar which will usually be denoted \(s\).</p>
<p>We will be playing with this structure a lot: forging transcripts \(R, e, s\); running multiple instances of a \(\Sigma\)protocol in parallel and performing logical operations on them. All of this will play out <em>mostly</em> in the form of a Schnorr signature; again, refer to previous blog posts or elementary explanations (including those written by me) for more on that.</p>
<h2>OR of \(\Sigma\)protocols, CDS 1994</h2>
<p>Let's start with the OR of \(\Sigma\)protocols. I <em>believe</em> this solution is due to <a href="https://link.springer.com/content/pdf/10.1007%2F3540486585_19.pdf">Cramer, Damgård and Schoenmakers '94</a></p>
<p>(Historical note: the "believe" is because I've seen it cited to that paper (which is famous for good reason, I guess); but in the paper they actually attribute <em>this specific idea</em> to "M. Ito, A. Saito, and T. Nishizeki: Secret Sharing Scheme realizing any Access Structure, Proc. Glob.Com. (1987)" ; unfortunately I can't find that on the 'net).</p>
<p>It is also described, with a brief discussion of its security proof, in <a href="https://crypto.stanford.edu/~dabo/cryptobook/BonehShoup_0_4.pdf">BonehShoup</a> Sec 19.7.2.</p>
<p>This is not, as far as I know, used at all(?) nor that widely discussed, but it is in some sense the most simple and logical way to get a 1 out of \(N\) ring signature; use the XOR (\(\oplus\)) operation:</p>
<p>We have in advance a set of public keys \(P_i\). We only know one private key for index \(j\), \(x_j\).</p>
<p>We'll now use a standard three move \(\Sigma\)protocol to prove knowledge of <strong>at least one key</strong> without revealing which index is \(j\).</p>
<p>We're going to fake the non\(j\)index signatures in advance. Choose \(s_i \stackrel{$}{\leftarrow} \mathbb{Z}_N\ ,\ e_i \stackrel{$}{\leftarrow} \mathbb{Z}_N \quad \forall i \neq j\).</p>
<p>Calculate \(R_i = s_iG  e_iP_i \quad \forall i \neq j\).</p>
<p>For the real signing index, \(k_j \stackrel{$}{\leftarrow} \mathbb{Z}_N\ ,\quad R_j = k_jG\).</p>
<p>We now have the full set of commitments: \((R_i \ \forall i)\)</p>
<p>Now for the clever part. In an interactive \(\Sigma\)protocol, we would at this point receive a random challenge \(e \in \mathbb{Z}_N\). For the Fiat Shamir transformed case, noninteractively (as for a signature), we use the constructed \(R\)values as input to a hash function, i.e. \(e = H(mR_i \ldots)\). We have already set the nonsigning index \(e\)values, for the signing index we set \(e_j = e \oplus (\bigoplus_{i \ne j}{e_i})\).</p>
<p>This allows us to calculate \(s_j = k_j + e_j x_j\), and we now have the full set of 'responses' for all the \(\Sigma\)protocols: \(s_i \ \forall i\). (but here we are using Fiat Shamir, so it's not actually a response).</p>
<p>By working this way we have ensured that the signature verifier can verify that the logical XOR of the three \(e\)values is equal to the Fiat Shamir based hashchallenge, e.g. for the case of three "signatures", we will have:</p>
<p>\(e = e_1 \oplus e_2 \oplus e_3 \stackrel{?}{=} H(mR_0R_1...)\)</p>
<p>where the verifier would calculate each \(R_i\) as \(s_iG  e_iP_i\).</p>
<p>The excellent feature of this of course is that it is perfectly hidden which of the three indexes was genuine. But the bad news is that the protocol as stated, used let's say as a signature scheme, requires about twice as many field elements as members of the group of signers. The verifier needs to be given \((s_1, \ldots s_n),(e_1 \ldots e_n)\).</p>
<p>Another excellent feature: this is not restricted to the Schnorr ID protocol. It can work with another identity protocol, and even better, it could work with a <em>mix</em> of them; they only have to share the one challenge \(e\).</p>
<h2>AbeOhkuboSuzuki (AOS) 2002 (broken version)</h2>
<p>This is an excellent <a href="https://www.iacr.org/cryptodb/archive/2002/ASIACRYPT/50/50.pdf">paper</a> generally, but its standout contribution, in this context, is a <strong>more compact</strong> version of the 1 of n ring signature above. To clarify here, both this and the previous are \(O(n)\) where \(n\) is the group size, so "much more compact" is about the constant factor (scale not scaling!); we reduce it from roughly 2 to roughly 1.</p>
<p>"Broken version"  here I'll present a slightly simpler form than the one in the paper, and then explain the serious problem with it  which I hope will be productive. <strong>Please don't mistake this as meaning that the AOS design was broken, it was never presented like this in the paper!</strong></p>
<p>Anyway, I think the best explanation for what's going on here conceptually is due to A. Poelstra in the <a href="https://github.com/Blockstream/borromean_paper">Borromean ring signatures paper</a>, particularly Section 2 ; the reference to time travel may seem whimsical but it gets to the heart of what's going on here; it's about having a simulated form of causality with one way functions, and then violating that.</p>
<p>In short: creating an ordinary Schnorr sig without the key (i.e. forging) is impossible because, working at the curve point level of the equation (\(sG = R + H(mR)P\)), you need to know the hash value before you can calculate \(R\), but you need to know the value of \(R\) before you can calculate the hash. So we see that two one way functions are designed to conflict with one another; only by removing one of them (going from curve points to scalar eqn: (\(s = k + H(mkG)x\)), can we now create a valid \(s, R, m\) set.</p>
<p>To achieve that goal over a set of keys, we can make that "simulated causality enforcement" be based on the same principle, but over a set of equations instead of one. The idea is to make the commitment \(H(mR)\) use the \(R\) value from the "previous" signer/key/equation, where "previous" is modulo \(N\), i.e. there is a loop of dependencies (a ring, in fact).</p>
<p><span style="textdecoration: underline;">Quick description:</span></p>
<p>Our goal is a list of \(N\) correctly verifying Schnorr signature equations, with the tweak as mentioned that each hashvalue refers to the "previous" commitment. We will work with \(N=4\) and index from zero for concreteness. Our goal is:</p>
<p>\(s_0 G = R_0 + H(mR_3)P_0\)</p>
<p>\(s_1 G = R_1 + H(mR_0)P_1\)</p>
<p>\(s_2 G = R_2 + H(mR_1)P_2\)</p>
<p>\(s_3 G = R_3 + H(mR_2)P_3\)</p>
<p>Again for concreteness, we imagine knowing specifically the private key \(x_2\) for index 2, only. We can successfully construct the above, but only in a certain sequence:</p>
<p>Choose \(k_2 \stackrel{$}{\leftarrow} \mathbb{Z}_N,\ R_2 = k_2G\), choose \(s_3 \stackrel{$}{\leftarrow} \mathbb{Z}_N\).</p>
<p>\(\Rightarrow R_3 = s_3 G  H(mR_2)P_3\). Now choose \(s_0 \stackrel{$}{\leftarrow} \mathbb{Z}_N\).</p>
<p>\(\Rightarrow R_0 = s_0 G  H(mR_3)P_0\). Now choose \(s_1 \stackrel{$}{\leftarrow} \mathbb{Z}_N\).</p>
<p>\(\Rightarrow R_1 = s_1 G  H(mR_0)P_1\).</p>
<p>Last, do not choose but <strong>calculate</strong> \(s_2\): it must be \(s_2 = k_2 + H(mR_1)x_2\).</p>
<p>After this set of steps, the set of data: \(e_0, s_0, s_1, s_2, s_3\) can be verified without exposing which private key was known. Here is the verification:</p>
<p>Given \(e_0, s_0\), reconstruct \(R_0 = s_0G e_0P_0\).</p>
<p>\(\Rightarrow e_1 =H(mR_0)\ ,\ R_1 = s_1 G  e_1P_1\)</p>
<p>\(\Rightarrow e_2 =H(mR_1)\ ,\ R_2 = s_2 G  e_2P_2\)</p>
<p>\(\Rightarrow e_3 =H(mR_2)\ ,\ R_3 = s_3 G  e_3P_3\)</p>
<p><strong>Check</strong>: \(e_0 \stackrel{?}{=} H(mR_3)\).</p>
<h3>Security weaknesses</h3>
<p>The description above can't be described as secure.</p>
<p>To give a hint as to what I mean: is there something <strong>not completely fixed</strong> in the above construction? Maybe an issue that's not even specific to the "ring" construction, but even for any one of the signature equations?</p>
<p>....</p>
<p></p>
<p></p>
<p>The answer is the keys, \(P_i\). We can in the most general case consider three scenarios, although there may be some gray areas between them:</p>
<ul>
<li>Key(s) fixed in advance: \(P_1 \ldots P_N\) are all specified before doing anything, and not allowed to change by the verifier. Every signature must be on that set of keys.</li>
<li>The <em>set </em><em>of possible keys</em> is fixed in advance exactly as described above, but the <em>set of keys used in the ring</em> is chosen by the signer, dynamically, in signing oracle queries or forgery attempts.</li>
<li>Even the set of possible keys is dynamic. That is to say, any valid curve point (for EC case) is a valid potential key in (ring) signature.</li>
</ul>
<p>This is not a full taxonomy of possible attack scenarios, either. Not only must we consider the difference between EUFCMA and SUFCMA as was discussed in the previous blog (a reminder: with SUF, a forger should not be able to even create a second signature on the same message  ECDSA doesn't have this in naive form), but much more: we must also consider which of the above three key settings applies.</p>
<p>Even outside of ring signature settings, just considering a large scale deployment of a signature scheme across millions or billions of keys, could mean that the difference between these cases really matters. In <a href="https://eprint.iacr.org/2015/996">this</a> paper by Dan Bernstein the term MUUFCMA is used to refer to the "multiuser" setting for this, where only singlekey signatures are used but one must consider whether having billions of other keys and signing oracles for them might impact the security of <strong>any one</strong> key (notice the huge difference between "I want to forge on \(P\)" and "I want to forge on any existing key" is, in this scenario).</p>
<p>So enough about settings, what exactly constitutes a security problem with the above version of the AOS ring sig?</p>
<p>Consider any one element in the ring like:</p>
<p>\(s_0 = R_0 + H(mR_3)P_0\)</p>
<p>where, for concreteness, I choose \(n=4\) and look at the first of 4 signature equations. Because of Schnorr's linearity (see <a href="https://joinmarket.me/blog/blog/flippingthescriptlessscriptonschnorr/">this earlier blog post</a> for some elucidations on the <em>advantage</em> of this linearity, although it was also noted there that it had concomitant dangers (worse, actually!)), there are two obvious ways we could tweak this equation:</p>
<p>(1) Tweaked \(s\) values on fixed message and tweaked keys:</p>
<p>Choose \(\alpha \in \mathbb{Z}_N\) and set \(s' = s_0 +\alpha\). We will not alter \(R=kG\), but we alter \(P_0 \rightarrow P_0 + e_0^{1}\alpha G\). This makes the verification still work <strong>without altering the fixing of the nonce in the hash value \(e_0\):</strong></p>
<p>\(s_0 G + \alpha G = R_0 + e_0 P_0 + \alpha G = R_0 + e_0\left(P_0 + e_0^{1}\alpha G\right)\)</p>
<p>So <strong></strong>it's really not clear how bad this failing is; it's <em>kinda</em> a failure of strong unforgeability, but that notion doesn't precisely capture it: we created a new, valid signature against a <span style="textdecoration: underline;">new</span> key, but with two severe limitations: we weren't able to alter the message, and also, we weren't able to <em>choose</em> the new key \(P'\). That last is slightly unobvious, but crucial : if I have a preprepared \(P^{*}\), I cannot choose \(\alpha\) to get \(P' = P^{*}\) as that would require a discrete logarithm break.<strong></strong></p>
<p>A final statement, hopefully obvious: the above can apply to any and all of the elements of the ring, so the forgery could consist of an entirely different and random set of keys, not related to the starting set; but the message would be the same, as would the \(R\) values.</p>
<p>(2) Completely different messages on tweaked keys, with the same signature</p>
<p>This one is almost certainly more important. Algebraically, we here allow alterations to the \(e\) values, using multiplication rather than addition:</p>
<p>Given the same starting \(s_0\) as in (1), we take a chosen new message \(m^{*}\) and calculate the new \(e^{*} = H(m^{*}R_3)\). If we likewise tweak the public key we get that \(s_0, R_0\) is a valid signature on the new message, with the tweaked key:</p>
<p>\(s_0 G = R_0 + e_0^{*}\left(\frac{e_0}{e_0^{*}} P_0\right)\)</p>
<p>We can see here that this produces a forgery with the same signature values (but different hash values) on the new keys.</p>
<p>Most definitions of security against forgery require the attacker to create a signature on a notpreviouslyqueried message  so this <em>is</em> a successful attack, by most measures.</p>
<p>However it does share the same limitation with (1) mentioned above  that you cannot "control" the keys on which you get a signature, unless you know a relative discrete log between one of the existing keys and your new key, which implies you knew the secret key of the first (in which case all this is pointless; whenever you have a private key, there is no forgery on it).</p>
<p><strong>All of this should make very clear the reason why the real AOS (see Section 5.1 of the paper) discretelog ring signature fixes the entire set of keys inside the hash, i.e. \(e_i = H(m  R_{(i1)\%n} P_0 \ldots P_{n1})\).</strong></p>
<h3>Key Prefixing</h3>
<p>The method in the previous bolded sentence is sometimes called "keyprefixing". One way of looking at it: the FiatShamir transform that takes the Identity Protocol into a signature scheme, should hash the conversation transcript between the prover and verifier, previous to the challenge step; by including the public keys in this hash, we are treating the keyset as part of the conversation transcript, rather than something exprotocolrun.</p>
<p>Also, the discussion above (both cases (1) and (2)) show clearly that the same weakness exists for a single (\(n=1\)) key case.</p>
<p><span style="textdecoration: underline;">And yet, for the single key case, it was not a done deal historically  this caused real world arguments!</span>. After all, there are many use cases where the key <em>is</em> a given exprotocolrun, plus there may be some practical disadvantage to doing the keyprefixing.</p>
<p>In <a href="https://rd.springer.com/chapter/10.1007%2F9783662530085_2">this</a> paper from CRYPTO2016, the controversy arising out of this is elucidated, showing that these theoretical concerns had very substantial impact on arguably the largest real world crypto usage (TLS):</p>
<blockquote>
<p>"Keyprefixing comes with the disadvantage that the entire publickey has to<br/>be available at the time of signing. Specifically, in a CFRG message from Sep<br/>tember 2015 Hamburg [32] argues “having to hold the public key along with<br/>the private key can be annoying” and “can matter for constrained devices”.<br/>Independent of efficiency, we believe that a cryptographic protocol should be<br/>as light as possible and prefixing (just as any other component) should only<br/>be included if its presence is justified. Naturally, in light of the GMLS proof,<br/>Hamburg [32] and Struik [44] (among others) recommended against key prefixing<br/>for Schnorr. Shortly after, Bernstein [10] identifies the error in the GMLS theo<br/>rem and posts a tight security proof for the keyprefixed variant of Schnorr signa<br/>tures. In what happens next, the participant of the CFRG mailing list switched<br/>their minds and mutually agree that keyprefixing should be preferred, despite of<br/>its previously discussed disadvantages. Specifically, Brown writes about Schnorr<br/>signatures that “this justifies a MUST for inclusion of the public key in the mes<br/>sage of the classic signature” [16]. As a consequence, keyprefixing is contained in<br/>the current draft for EdDSA [33]..."</p>
</blockquote>
<p><em>Technical note: the "GMLS proof" mentioned in the above is the proof given in <a href="https://www.researchgate.net/publication/256720499_Public_key_signatures_in_the_multiuser_setting">this</a> paper, that was intended to reduce the security of the multiuser setting to that of the singleuser setting, and that Dan Bernstein's <a href="https://eprint.iacr.org/2015/996">paper</a> previously mentioned proved to be invalid.</em></p>
<p><em></em></p>
<p>What's the TLDR? Fix the keys in any group/ring/multisignature. And even that may not be enough, see <a href="https://eprint.iacr.org/2018/068">MuSig</a> for details of why it really isn't, in the scenario of Bitcoin aggregated multisig.</p>
<h2>Borromean, MaxwellPoelstra 2015</h2>
<p>I covered this extensively (including description of AOS as above) in my <a href="https://github.com/AdamISZ/ConfidentialTransactionsDoc/">CT writeup</a> section 3.2</p>
<p>The idea of the construction as outlined in <a href="https://github.com/Blockstream/borromean_paper">the paper by Maxwell, Poelstra</a> is to increase the spaceefficiency of the published proof even more. By having several ring signatures joined at a single index we get a reduction in the number of \(e\) values we publish. This is basically the same idea as the "AND of \(\Sigma\)protocols" discussed a little later in this document (although here we will only be using it for achieving a specific goal, "Linkability", see more on this next).</p>
<p>For the real world context  Borromean ring signatures are used in certain implementations of Confidential Transactions (e.g. Liquid by Blockstream) today, and were previously used also in Monero for the same goal of CT. They are a radically different usecase of ring signatures to the one mostly described in the below; instead of using a ring signature to hide the identity of a signer, they are used to hide which exponent contains values in the encoding of a value committed to in a Pedersen commitment. This allows arithmetic to be done on the Pedersencommitted amount without worrying about overflow into negative values modulo \(N\).</p>
<h2>Linkability and Exculpability</h2>
<p>In this section we'll briefly describe certain key features that turn out to be useful in some realworld applications of a ring signature, before in the following sections laying out how these features are, or are not, achieved.</p>
<h3>Linkability (and spontaneity)</h3>
<p>At first glance, the idea "linkability" with a ring signature seems to be a contradiction. Since we are trying to achieve signer ambiguity/anonymity, we don't really want any "linking" being done. But the idea is rather clever, and proves to be very interesting for digital cash.</p>
<p>In a <strong>linkable</strong> ring signature, a participant with key \(P \in L\) (i.e. \(L\) is a particular set of public keys), should be able to produce one ring signature on a given message, but should not be able to do so again without the two ring signatures being linked. Thus, functionally, each participant can only make such a signature once (note: they can still retain anonymity if doublesigning).</p>
<p>This restrictiontoonesignaturewhilekeepinganonymity is easily seen to be valuable in cases like electronic voting or digital cash, as well as the oftcited example explained in the next paragraph.</p>
<p>The <strong>spontaneity</strong> property should be a lot more obvious. Consider the example of a whistleblower. We would want individuals in some large group (e.g. government bureaucrats) to attest to a statement, while only revealing group membership and not individual identity. Clearly this is not workable if it requires cooperation of other members of the group (even in any setup phase), so it's necessary that the individual can create the ring signature "spontaneously", knowing only the public key of other participants.</p>
<p>The paper uses the abbreviation LSAG for this type of signature: "Linkable Spontaneous Anonymous Group" signature.</p>
<p>Note that the previous two constructions (CDS, AOS) can also have this spontaneity property; but not the linkability property.</p>
<h3>Culpability, Exculpability and Claimability</h3>
<p>A ring signature can be described as exculpable if, even given knowledge of the signing private key, an adversary cannot deduce that that signing key was the one used to create the ring signature.</p>
<p>Notice that such a property may be immensely important in a range of scenarios where a ring sig is useful  e.g. for a whistleblower whose cryptographic keys were stolen or extracted by force, he could still plausibly deny being the origin of a leak.</p>
<p>The reader can easily verify that the AOS construction, for example, has this exculpability. The fact that a particular key is released e.g. \(x_2\) in our concrete example, does not allow inference of it having been used to create that signature. Any other key could have created the signature, using the same signing algorithm.</p>
<p>The LWW LSAG, which we'll describe shortly, is on the other hand <strong>culpable</strong>, i.e. the opposite  because the key image can be verified to be tied to one particular key.</p>
<p>It's easy to see that the two properties <strong>exculpability</strong> and <strong>linkability</strong> are somewhat in conflict, although I'm not aware of a theorem that <em>absolutely requires</em> linkability to somehow tag one key in case it is leaked.</p>
<p>Lastly, I'll mention <strong>claimability</strong>, which is briefly described also in the LWW paper (see below). It may be possible for the owner of a key to independently/voluntarily prove that they were the source of a given ring signature, which doesn't logically require culpability. Claimability is generally easy to achieve with some proof of knowledge technique.</p>
<h2>AND of \(\Sigma\)protocols, DLEQ</h2>
<p>The thoughtful reader probably won't have much trouble in imagining what it would mean to do the logical AND of 2 \(\Sigma\)protocols.</p>
<p>"AND" here just means you need to prove to the Verifier that you know both secrets / both conditions are true. So this only requires that you can answer both challenges (second step) with correct responses. Using the standard notation, that means generating two transcripts:</p>
<p>\((R_1, e, s_1) \quad (R_2, e, s_2)\)</p>
<p>i.e. the same \(e\)value is given to both protocol runs after receiving the initial commitments from each. FiatShamirising this protocol will work the same as the usual logic; if considering a signature scheme, we'll be hashing something like \(H(mR_1R_2P_1P_2)\), if we include, as we have learnt to, keyprefixing.</p>
<p>As we already mentioned, the Borromean ring signature design uses this idea to compactify a set of ring signatures, since only one \(e\)value is being published, rather than \(M\) for \(M\) ring signatures.</p>
<p>This much is not superinteresting; but we can tighten this up a bit and only use <strong>one</strong> commitment and response in a special case:</p>
<h3>Proof of Discrete Log Equivalence (DLEQ, PoDLE)</h3>
<p>See one of the first posts on this <a href="https://joinmarket.me/blog/blog/poodle">blog</a> for a description of this technique; here we're giving a slightly deeper look at the meaning.</p>
<p>If you are proving not only knowledge of a secret \(x\), but also that two curve points have the same discrete log \(x\) w.r.t. different bases \(G\) and \(J\) (whose relative discrete log must not be known; see earlier blog post etc.), you can condense the above AND by reusing the commitment and challenge for the two bases:</p>
<p>\(\mathbb{P} \rightarrow \mathbb{V}\): \(R_1= kG,R_2=kJ\)</p>
<p>\(\mathbb{V} \rightarrow \mathbb{P}\): \(e = H(mR_1R_2P_1P_2)\)</p>
<p>\(\mathbb{P} \rightarrow \mathbb{V}\): \(s\), (in secret: \(=k+ex\))</p>
<p>Now, if the prover acted honestly, his construction of \(s\) will correctly pass verification <strong>twice</strong>:</p>
<p>\(sG \stackrel{?}{=}R_1 +e P_1 \quad sJ \stackrel{?}{=} R_2 + eP_2\)</p>
<p>... and notice that it would be impossible to make that work for different \(x\)values on the two bases \(G\) and \(J\) because you would need to find \(k_1, k_2 \in \mathbb{Z}_N, x_1, x_2 \in \mathbb{Z}_N\) such that, <strong>without knowing \(e\) in advance, </strong>\(s = k_1 + ex_1 =k_2 + ex_2\), which is clearly impossible.</p>
<p>Proof of soundness is easy to see using the standard rewinding technique (see e.g. previous blog post amongst many other places); after the two upfront commitments are fixed and the \(e\)values are "forked", we will get two \(s\) values as usual and extract \(x\).</p>
<h2>LiuWeiWong 2004 LSAG</h2>
<p>Shortly after the AOS paper, Liu, Wei and Wong published a <a href="https://www.researchgate.net/publication/220798466_Linkable_Spontaneous_Anonymous_Group_Signature_for_Ad_Hoc_Groups_Extended_Abstract">paper</a> outlining how the same basic idea could be extended to a slightly more complex context of requiring <strong>linkability</strong>, as earlier mentioned. It uses a combination of the above: DLEQ via AND of \(\Sigma\)protocols, and OR of \(\Sigma\)protocols for the ring signature hiding effect. Detailed algorithm with commentary follows.</p>
<h3>LiuWeiWong's LSAG algorithm</h3>
<p>We start with a keyset \(L = \{P_0 \ldots P_{n1}\}\) chosen by the signer, whose index will be \(\pi\) (note the ambiguities about "what is the set of valid keys?" as was discussed under "Key Prefixing"). We then form a special new kind of curve point that we'll name from now on as the <strong>key image</strong> (for reasons that'll become clear):</p>
<p>\(I =x_{\pi} \mathbb{H}(L)\)</p>
<p>Here \(\mathbb{H}\) is a hash function whose output space is points on the curve, rather than scalar numbers. (<em>The mechanical operation for doing this is sometimes described as "coerce to point"; for example, take the 256 bit number output by SHA256 and interpret it as an \(x\)coordinate on secp256k1, find the "next" valid point \(x,y\), incrementing \(x\) if necessary, or whatever; just has to be deterministic</em>). \(\mathbb{H}(L)\) is therefore going to play the same role as \(J\) in the previous section, and we assume intractability of relative discrete log due to the hashing.</p>
<h3>Signing LWW LSAG</h3>
<p>The following steps are very similar "in spirit" to AOS; we still "extend the causality loop" (bastardising Poelstra's description) over the whole set of signatures instead of just one, but this time we also "lift" the loop onto a base of \(\mathbb{H}(L)\) and replicate the signatures there, too:</p>
<ul>
<li>Set \(k_{\pi} \stackrel{$}{\leftarrow} \mathbb{Z}_N\)</li>
<li>Form the hashchallenge at the next index: \(e_{\pi+1} = H(mLk_{\pi}Gk_{\pi}\mathbb{H}(L)I)\)</li>
<li>Note to the above: \(k_{\pi}G\) was previously called \(R_{\pi}\) in AOS; we are trying to preserve here, the same notation where possible; and of course it's the \(R\) value, not the \(k\)value that will be known/calculated by the verifier. The same applies to the "lifted" noncepoint which follows it in the concatenation. With respect to the key image, note that it <em>will</em> be published and known to the verifier; but he won't know which index it corresponds to.</li>
<li>Pick \(s_{\pi+1} \stackrel{$}{\leftarrow} \mathbb{Z}_N\); then we do as in AOS, but duplicated; we set:</li>
<li>\(R_{\pi+1} = s_{\pi+1}G  e_{\pi+1}P_{\pi+1}\) and \(R^{*}_{\pi+1} = s_{\pi+1}\mathbb{H}(L)  e_{\pi+1}I\)</li>
<li>I realise the last line is pretty dense, so let's clarify: the first half is exactly as for AOS; calculate \(R\) given the random \(s\) and the justcalculated hash value \(e\). The <em>second</em> half is <strong>the same thing with the base point \(G\) replaced with \(\mathbb{H}(L)\), and the pubkey replaced with \(I\) at every index</strong>. We used a shorthand \(R^{*}\) to mean \(k_{\pi+1}\mathbb{H}(L)\), because of course we don't actually <em>know </em>the value \(k_{\pi+1}\).</li>
<li>Calculate the next hashchallenge as \(e_{pi+2} = H(mLR_{\pi+1}R^{*}_{\pi+1}I)\)</li>
<li>Etc...</li>
<li>As with AOS, we can now forge all the remaining indices, wrapping around the loop, by repeating the above operation, generating a new random \(s\) at each step, until we get back to the signing index \(\pi\), when we must calculate \(s_{\pi}\) as: \(s_{\pi} = k_{\pi} + e_{\pi}x_{\pi}\).</li>
<li>Signature is published as \(\sigma_{L}(m) = (s_0 \ldots s_{n1}, e_0, I)\). (As before, if the keyset \(L\) is not specified in advance, it will have to be published for the verifier).</li>
</ul>
<p>So what we're doing here is OR(DLEQ(0), DLEQ(1),.... DLEQ(n1)). And as observed, each DLEQ is actually an AND: "AND(I know x for P, x for P is same as x for P2)". Hence this represents a clever combination of AND and OR of \(\Sigma\)protocols.</p>
<p><em>On a personal note, when I first saw something of this type (I think it was Cryptonote, see below), I found it quite bewildering, and I'm sure I'm not alone! But what partially saved me is having already studied PoDLE/DLEQ as well as AOS ring sigs, so I could intuit that something combining the two ideas was going on. I hope the previous painstaking introductions make it all a lot clearer!</em></p>
<p>Note the key similarities and difference(s) in the published signature, to the AOS case: you still only need to publish one hash \(e_0\) since the others are determined by it, but you <strong>must</strong> publish also the key image \(I\); if another LWW LSAG is published using the same private key, it will perforce have the same key image, and be recognized as having come from the same key <span style="textdecoration: underline;">without revealing which key</span>.</p>
<p>The protocol using the LSAG can thus reject a "doublesign", if desired.</p>
<p>Let's sanity check that we understand the verification algorithm, since it is slightly different than AOS:</p>
<h3>Verifying LWW LSAG</h3>
<p>Start with the given keyset \(L\), the message \(m\) and the signature \((s_0 \ldots s_{n1}, e_0, I)\)</p>
<p></p>
<ul>
<li>Construct \(e_{1} = H(mLR_{0}R^{*}_{0}I)\) using \(R_0 = s_0G  e_0 P_0\) and \(R^{*}_{0} = s_0 \mathbb{H}(L)  e_0 I \)</li>
<li>Repeat at each index using the new \(e_j\) until \(e_0\) is calculated at the last step and verify it matches: \(e_0 \stackrel{?}{=} H(mLR_{n1}R^{*}_{n1}I)\). Accept if so, reject if not.</li>
</ul>
<p>(with the additional point mentioned: the protocol using the sig scheme may also reject this as valid if \(I\) has already been used; this additional protocol step is usually described as "LINK" in the literature).</p>
<h3>A brief note on the key image</h3>
<p>Make sure you get the difference between this \(\mathbb{H}(L)\) and the previous \(J\) as per the general DLEQ. In the latter case we can (and should) choose an arbitrary globallyagreed NUMS point, for example hashing the standard curve base point \(G\) (with the "coercetopoint" technique mentioned). In this case, we have chosen something that both signer and verifier agree on, as part of the <strong>setting</strong> of this particular run of the protocol  it's deterministically tied to the keyset \(L\). The key image\(I\) is analogous to \(P_2\) in my PoDLE blog post; it's the signer's "hidden", onetime key.</p>
<p>This changes in the next construction, Back 2015. But first, a few words on security.</p>
<h2>Security arguments for the LWW LSAG</h2>
<p>The general approach to proving <strong>unforgeability </strong>of this ring signature is the same as that for the basic Schnorr signature as described in the previous blog post.</p>
<p>A wrapper around an attacker \(\mathbb{A}\) who we posit to have the ability to construct a forgery without knowing any private key \(x_i\), will, as before, have to guess which random oracle query corresponds to the forgery, and will want to provide two different "patched" answers to the RO query at that point. As before, there will be some reduced probability of success due to having to make this kind of guess, and so the reduction will be even less tight than before.</p>
<p>Also as before, in the EUFCMA model, we must allow for an arbitrary number of signing oracle as well as RO queries, which complicates the statistical analysis considerably, but the basic principles remain the same. If at some point forgery is successfully achieved twice at the same index, we will have something like:</p>
<p>\(x_{\pi} = \frac{s^{*}_{\pi}s_{\pi}}{e^{*}_{\pi}e_{\pi}}\)</p>
<p>where the * superscripts indicate the second run, and the \(e\)values being the patched RO responses.</p>
<p>And as usual, with appropriate statistical arguments, one can generate a reduction such that forgery ability with a certain probability \(p\) implies thus ability to solve ECDLP with a related probability \(p'\).</p>
<p>For proving <strong>signer ambiguity </strong> for simplicity, we break this into two parts. If <em>all</em> of the private keys are known to the attacker (e.g. by subpoena), then this property completely fails. This is what we called <strong>culpability</strong>. It's easy to see why  we have the key image as part of the signature, and that is deterministically reproducible given the private key. If <em>none </em>of the private keys are known to the attacker, the problem is reduced to the <strong>solution of the <a href="https://en.wikipedia.org/wiki/Decisional_Diffie%E2%80%93Hellman_assumption">Decisional Diffie Hellman Problem</a></strong>, which is considered computationally hard. The reduction is quite complicated, but as in a standard zero knowledgeness proof, the idea is that a Simulator can generate a transcript that's statistically indistinguishable from a genuine transcript.</p>
<p>For proving <strong>linkability<em> </em></strong> in the LWW paper an argument is made that this reduces to ECDLP in more or less the same was as for the unforgeability argument, using two pairs of transcripts for two different signatures which are posited to be based on the same private key but having different key images. Examination of the two pairs of transcripts allows one to deduce that the private key in the two cases are the same, else ECDLP is broken.</p>
<p></p>
<p>Notice that these security arguments are <span style="textdecoration: underline;">much more complicated than for the single Schnorr signature case</span> and perhaps for two distinct reasons: one, because the ring signature is a more complex algebraic construction, with more degrees of freedom, but also, because we are asking for a significantly richer set of properties to hold. In particular notice that even for unforgeability, the EUFCMA description is not good enough (we've already discussed this a bit); we need to consider what happens when creating multiple signatures on different keysets and how they overlap. Signer anonymity/ambiguity is especially difficult for LWW and its postdecessors (see below), because by design it has been weakened (culpability).</p>
<h2>Back 2015; compression, singleuse</h2>
<p><em>This is a good place to note that the constructions starting with LWW are described in some detail in the useful document <a href="https://ww.getmonero.org/library/ZerotoMonero100.pdf">ZeroToMonero</a><strong></strong></em>.</p>
<p>Adam Back <a href="https://bitcointalk.org/index.php?topic=972541.msg10619684#msg10619684">posted</a> in 2015 on bitcointalk about a potential space saving over the cryptonote ring signature, based on using AOS and tweaking it to include a key image.</p>
<p>As was noted above, it's a space saving of asymptotically about 50% to use a scheme like AOS that only requires publication of one hash challenge as opposed to one for each index (like the CDS for example).</p>
<p>He then followed up noting that a very similar algorithm had already been published, namely the LWW we've just described in the above, and moreover it was published three years before FujisakiSuzuki that was the basis of cryptonote (see below). So it was <em>somewhat</em> of an independent rediscovery, but there is a significant tweak. I'll outline the algorithm below; it'll look very similar to LWW LSAG, but there's a difference.</p>
<h3>Signing BackLSAG</h3>
<ul>
<li>Define key image<strong></strong> \(I =x_{\pi}\mathbb{H}(P_{\pi})\);</li>
<li>Set \(k_{\pi} \stackrel{$}{\leftarrow} \mathbb{Z}_N\)</li>
<li>Form the hashchallenge at the next index: \(e_{\pi+1} = H(mk_{\pi}Gk_{\pi}\mathbb{H}(P_{\pi}))\)</li>
<li>Pick \(s_{\pi+1} \stackrel{$}{\leftarrow} \mathbb{Z}_N\); then:</li>
<li>\(R_{\pi+1} = s_{\pi+1}G  e_{\pi+1}P_{\pi+1}\) and \(R^{*}_{\pi+1} = s_{\pi+1}\mathbb{H}(P_{\pi+1})  e_{\pi+1}I\)</li>
<li>Calculate the next hashchallenge as \(e_{pi+2} = H(mR_{\pi+1}R^{*}_{\pi+1})\)</li>
<li>Etc...</li>
<li>As with AOS and LWW, we can now forge all the remaining indices, wrapping around the loop, by repeating the above operation, generating a new random \(s\) at each step, until we get back to the signing index \(\pi\), when we must calculate \(s_{\pi}\) as: \(s_{\pi} = k_{\pi} + e_{\pi}x_{\pi}\).</li>
<li>Signature is published as \(\sigma_{L}(m) = (s_0 \ldots s_{n1}, e_0, I)\), as in LWW (\(L\) being the set of \(P\)s).</li>
</ul>
<p>Verification for this is nearidentical as for LWW, so is left as an exercise for the reader.</p>
<h3>What's the difference, and what's the purpose?</h3>
<p>The tweak  which is very similar to Cryptonote (makes sense as it was an attempt to improve that)  is basically this: by making each of the signatures in the shifted base point version symmetrical (example: \(s_2 \mathbb{H}(P_2) = k_2 \mathbb{H}(P_2) + e_2 I\)), it means that a key image will be valid <em>independent of the set of public keys, \(L\).<strong></strong></em> This is crucial in a cryptocurrency application  we need the key image to be a unique double spend signifier across many different ring signatures with different keysets  the keys are ephemeral and change between transactions.</p>
<p>So it's a blend of the LWW LSAG, which has the advantage of space compaction for the same reason as AOS  only one hash must be published, the others can be deduced from the ring structure  with the FS2007/Cryptonote design, which fixes the key image to the key and not just the specific ring.</p>
<p>However I have to here leave open whether the security arguments of LWW carry across to this case. I note that the original description did<em> not </em>include the keyset in the hash challenge (notice absence of \(L\)); but see the note on MLSAG below.</p>
<h2>FujisakiSuzuki 2007 and Cryptonote</h2>
<p><a href="https://cryptonote.org/whitepaper.pdf">Cryptonote </a>was adapted from a paper of <a href="https://eprint.iacr.org/2006/389.pdf">Fujisaki and Suzuki</a> describing an alternate version of a linkable (here "traceable") ring signature, in 2007. We won't dwell on these constructions here (except inasmuch as we referred to them above), as they provide the same linkability function as the above LSAG, but are less compact. Instead, in the final section, I'll describe how Monero has applied LWW LSAG and the Back LSAG to their specific requirements.</p>
<h2>Monero MLSAG</h2>
<p>For anyone paying close attention all the way through, there will be nothing surprising here!</p>
<p>For a cryptocurrency, we build transactions consisting of multiple inputs. Each input in Monero's case uses a ring signature, rather than a single signature, to authenticate the transfer, referring back to multiple pubkeys possessing coins as outputs of earlier transactions.</p>
<p>So here we need <strong>one ring signature per input</strong>. Moreover, per normal transaction logic, we obviously need <em>all</em> of those ring signatures to successfully verify. So this is another case for the "AND of \(\Sigma\)protocols". We just run \(M\) cases of Back's LSAG and combine them with a single \(e\) hash challenge at each key index (so the hash challenge kind of "spans over the inputs"). Additionally, note that the hash challenge here is assumed to include the keyset with a generic \(L\) (limiting tiresome subscripting to a minimum...).</p>
<p>To sign \(M\) inputs each of which have \(n\) keys:</p>
<ul>
<li>For each input, define key image<strong></strong> \(I_i =x_{i,\pi}\mathbb{H}(P_{i,\pi}) \ \forall i \in 0 \ldots M1\);</li>
<li>Set \(k_{i, \pi} \stackrel{$}{\leftarrow} \mathbb{Z}_N \ \forall i \in 0 \ldots M1\)</li>
<li>Form the hashchallenge at the next index: \(e_{\pi+1} = H(mLk_{0, \pi}Gk_{0, \pi}\mathbb{H}(P_{0,\pi})k_{1, \pi}Gk_{1, \pi}\mathbb{H}(P_{1,\pi}) ...)\)</li>
<li>Pick \(s_{i, \pi+1} \stackrel{$}{\leftarrow} \mathbb{Z}_N\ \forall i \in 0 \ldots M1\); then:</li>
<li>\(R_{i, \pi+1} = s_{i, \pi+1}G  e_{\pi+1}P_{i, \pi+1}\) and \(R^{*}_{i, \pi+1} = s_{i, \pi+1}\mathbb{H}(P_{i, \pi+1})  e_{\pi+1}I_i \ \forall i \in 0 \ldots M1\)</li>
<li>Calculate the next hashchallenge as \(e_{\pi+2} = H(mLR_{0, \pi+1}R^{*}_{0,\pi+1}R_{1, \pi+1}R^{*}_{2,\pi+1} ...)\)</li>
<li>Etc...</li>
<li>Logic as for AOS, LWW but duplicated at every input with single \(e\)challenge, and at signing index for all inputs (\(\pi\)): \(s_{i, \pi} = k_{i, \pi} + e_{i, \pi}x_{i, \pi}\ \forall i \in 0 \ldots M1\).</li>
<li>Signature is published as \(\sigma_{L}(m) = (s_{0,0} \ldots s_{0,M1}, \ldots, s_{n1,0}, \ldots s_{n1,M1}, e_0, I_0 \ldots I_{M1})\).</li>
</ul>
<p>Note:</p>
<p>(1) This algorithm as described requires each input to have the genuine signer at the same keyindex in the set of pubkeys for each input, which is a limitation.</p>
<p>(2) Monero has implemented Confidential Transactions, and this is folded in with the above into a new design which seems to have two variants "RingCTFull" and "RingCTSimple". This can be investigate further in the documents on RingCT as referenced in the previously mentioned <a href="https://ww.getmonero.org/library/ZerotoMonero100.pdf">ZeroToMonero</a>.</p>
<p></p>Liars, cheats, scammers and the Schnorr signature20190216T12:46:36+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/liarscheatsscammersandtheschnorrsignature/<p>How sure are <em>you</em> that the cryptography underlying Bitcoin is secure? With regard to one future development of Bitcoin's crypto, in discussions in public fora, I have more than once confidently asserted "well, but the Schnorr signature has a security reduction to ECDLP". Three comments on that before we begin:<em></em></p>
<ul>
<li>If you don't know what "reduction" means here, fear not, we will get deeply into this here.</li>
<li>Apart from simply <em>hearing</em> this and repeating it, I was mostly basing this on a loose understanding that "it's kinda related to the soundness proof of a sigma protocol" which I discussed in my <a href="https://github.com/AdamISZ/from0k2bp">ZK2Bulletproofs</a> paper, which is true  but there's a lot more involved.</li>
<li>The assertion is true, but there are caveats, as we will see. And Schnorr is different from ECDSA in this regard, as we'll also see, at the end.</li>
</ul>
<p>But why write this out in detail? It actually came sort of out of left field. Ruben Somsen was asking on slack about some aspect of Monero, I forget, but it prompted me to take another look at those and other ring signatures, and I realised that attempting to understand the <strong>security</strong> of those more complex constructions is a nonstarter unless you <strong>really understand why we can say "Schnorr is secure" in the first place</strong>.</p>
<h3>Liars and cheats</h3>
<p>The world of "security proofs" in cryptography appears to be a set of complex stories about liars  basically made up magic beans algorithms that <em>pretend</em> to solve things that nobody <em>actually </em>knows how to solve, or someone placing you in a room and resetting your clock periodically and pretending today is yesterday  and cheats, like "let's pretend the output of the hash function is \(x\), because it suits my agenda for it to be \(x\)" (at least in this case the lying is consistent  the liar doesn't change his mind about \(x\); that's something!).</p>
<p>I hope that sounds crazy, it mostly really is :)</p>
<p>(<em>Concepts I am alluding to include: the random oracle, a ZKP simulator, extractor/"forking", an "adversary" etc. etc.</em>)</p>
<h2>Preamble: the reluctant Satoshi scammer</h2>
<p>The material of this blog post is pretty abstract, so I decided to spice it up by framing it as some kind of scifi :)</p>
<p><img alt="" height="174" src="https://joinmarket.me/static/media/uploads/cube250082_6402.png" width="174"/></p>
<p>Imagine you have a mysterious small black cube which you were given by an alien that has two slots you can plug into to feed it input data and another to get output data, but you absolutely can't open it (so like an Apple device, but more interoperable), and it does one thing only, but that thing is astonishing: if you feed it a message and a <strong>public</strong> key in its input slot, then it'll <em>sometimes</em> spit out a valid Schnorr signature on that message.</p>
<p>Well in 2019 this is basically useless, but after considerable campaigning (mainly by you, for some reason!), Schnorr is included into Bitcoin in late 2020. Delighted, you start trying to steal money but it proves to be annoying.</p>
<p>First, you have to know the public key, so the address must be reused or something similar. Secondly (and this isn't a problem, but is weird and will become relevant later): the second input slot is needed to pass the values of the hash function sha2 (or whatever is the right one for our signature scheme) into the black box for any data it needs to hash. Next problem: it turns out that the device only works if you feed it a few <em>other</em> signatures of other messages on the same public key, first. Generally speaking, you don't have that. Lastly, it doesn't <em>always</em> work for any message you feed into it (you want to feed in 'messages' which are transactions paying you money), only sometimes.</p>
<p>With all these caveats and limitations, you fail to steal any money at all, dammit!</p>
<p>Is there anything else we can try? How about we pretend to be someone else? Like Satoshi? Hmm ...</p>
<p>For argument's sake, we'll assume that people use the Schnorr Identity Protocol (henceforth SIDP), which can be thought of as "Schnorr signature without the message, but with an interactive challenge". We'll get into the technicals below, for now note that a signature doesn't prove anything about identity (because it can be passed around), you need an interactive challenge, a bit like saying "yes, give me a signature, but *I* choose what you sign".</p>
<p>So to get people to believe I'm Satoshi (and thus scam them into investing in me perhaps? Hmm sounds familiar ...) I'm going to somehow use this black box thing to successfully complete a run of SIDP. But as noted it's unreliable; I'll need a bunch of previous signatures (let's pretend that I get that somehow), but I *also* know this thing doesn't work reliably for every message, so the best I can do is probably to try to <strong>scam 1000</strong> <strong>people simultaneously</strong>. That way they might reasonably believe that their successful run represents proof; after all it's supposed to be <em>impossible</em> to create this kind of proof without having the private key  that's the entire idea of it! (the fact that it failed for other people could be just a rumour, after all!)</p>
<p>So it's all a bit contrived, but weirder scams have paid off  and they didn't even use literally alien technology!</p>
<p>So, we'll need to read the input to our hash function slot from the magic box; it's always of the form:</p>
<p><code>message  Rvalue</code></p>
<p>... details to follow, but basically \(R\) is the starting value in the SIDP, so we pass it to our skeptical challenger(s). They respond with \(e\), intended to be completely random to make our job of proving ID as hard as possible, then <strong>we trick our black box</strong>  we don't return SHA2(\(mR\)) but instead we return \(e\). More on this later, see "random oracle model" in the below. Our magic box outputs, if successful, \(R, s\) where \(s\) is a new randomlooking value. The challenger will be amazed to see that:</p>
<p>\(sG = R + eP_{satoshi}\)</p>
<p>is true!! And the millions roll in.</p>
<p></p>
<p>If you didn't get in detail how that scam operated, don't worry, we're going to unpack it, since it's the heart of our technical story below. The crazy fact is that <strong>our belief that signatures like the Schnorr signature (and ECDSA is a cousin of it) is mostly reliant on basically the argument above.</strong></p>
<p>But 'mostly' is an important word there: what we actually do, to make the argument that it's secure, is stack that argument on top of at least 2 other arguments of a similar nature (using one algorithm as a kind of 'magic black box' and feeding it as input to a different algorithm) and to relate the digital signature's security to the security of something else which ... we <em>think</em> is secure, but don't have absolute proof.</p>
<p>Yeah, really.</p>
<p>We'll see that our silly scifi story has <em>some</em> practical reality to it  it really <em>is</em> true that to impersonate is a bit more practically feasible than to extract private keys, and we can even quantify this statement, somewhat.</p>
<p>But not the magic cube part. That part was not real at all, sorry.</p>
<h2>Schnorr ID Protocol and signature overview</h2>
<p>I have explained SIDP with reference to core concepts of Sigma Protocols and Zero Knowledge Proofs of Knowledge in Section 3.2 <a href="https://github.com/AdamISZ/from0k2bp">here</a> . A more thorough explanation can be found in lots of places, e.g. Section 19.1 of <a href="https://crypto.stanford.edu/~dabo/cryptobook/">Boneh and Shoup</a>. Reviewing the basic idea, cribbing from my own doc:</p>
<p></p>
<p>Prover \(\mathbf{P}\) starts with a public key \(P\) and a corresponding private key \(x\) s.t. \(P = xG\).<br/>\(\mathbf{P}\) wishes to prove in zero knowledge, to verifier \(\mathbf{V}\), that he knows \(x\).<br/>\(\mathbf{P}\) → \(\mathbf{V}\): \(R\) (a new random curve point, but \(\mathbf{P}\) knows \(k\) s.t. \(R = kG\))<br/>\(\mathbf{V}\) → \(\mathbf{P}\): \(e\) (a random scalar)<br/>\(\mathbf{P}\) → \(\mathbf{V}\): \(s\) (which \(\mathbf{P}\) calculated from the equation \(s = k + ex\))<br/>Note: the transcript of the conversation would here be: \((R, e, s)\).<br/>Verification works fairly trivially: verifier checks sG \(\stackrel{?}{=} R+eP\). See previously mentioned doc for details on why this is supposedly <em>zero knowledge</em>, that is to say, the verifier doesn't learn anything about the private key from the procedure.</p>
<p>As to why it's sound  why does it really prove that the Prover knows \(x\), see the same doc, but in brief: if we can convince the prover to rerun the third step with a modified second step (but the same first step!), then he'll be producing a second signature \(s'\) on a second random \(e'\), but with the same \(k\) and \(R\), thus:</p>
<p>\(x = \frac{ss'}{ee'}\)</p>
<p>So we say it's "sound" in the specific sense that only a knowerofthesecretkey can complete the protocol. But more on this shortly!</p>
<p>What about the famous "Schnorr signature"? It's just an noninteractive version of the above. There is btw a brief summary in <a href="https://joinmarket.me/blog/blog/flippingthescriptlessscriptonschnorr/">this</a> earlier blog post, also. Basically replace \(e\) with a hash (we'll call our hash function \(H\)) of the commitment value \(R\) and the message we want to sign \(m\):</p>
<p>\(e = H(mR)\)</p>
<p>; as mentioned in the justlinked blog post, it's also possible to add other stuff to the hash, but these two elements at least are necessary to make a sound signature.</p>
<p>As was noted in the 80s by <a href="https://link.springer.com/content/pdf/10.1007%2F3540477217_12.pdf">Fiat and Shamir</a>, this transformation is generic to any zeroknowledge identification protocol of the "three pass" or sigma protocol type  just use a hash function to replace the challenge with H(message, commitment) to create the new signature scheme.</p>
<p>Now, if we want to discuss security, we first have to decide what that even means, for a signature scheme. Since we're coming at things from a Bitcoin angle, we're naturally focused on preventing two things: forgery and key disclosure. But really it's the same for any usage of signatures. Theorists class security into at least three types (usually more, these are the most elementary classifications):</p>
<ul>
<li>Total break</li>
<li>Universal forgery</li>
<li>Existential forgery</li>
</ul>
<p>(Interesting historical note: this taxonomy is due to Goldwasser, Micali and Rackoff  the same authors who introduced the revolutionary notion of a "Zero Knowledge Proof" in the 1980s.)</p>
<p>Total break means key disclosure. To give a silly example: if \(k=0\) in the above, then \(s = ex\) and, on receipt of \(s\), the verifier could simply multiply it by the modular inverse of \(e\) to extract the private key \(x\). A properly random \(k\) value, or 'nonce', as explained ad nauseam elsewhere, is critical to the security. Since this is the worst possible security failure, being secure against it is considered the weakest notion of "security" (note this kind of "reverse" reasoning, it is very common and important in this field).</p>
<p>The next weakest notion of security would be security against universal forgery  the forger should not be able to generate a signature on any message they are given. We won't mention this too much; we will focus on the next, stronger notion of "security":</p>
<p>"Security against existential forgery under adaptive chosen message attack", often shortened to EUFCMA for sanity (the 'adaptive(ly)' sometimes seems to be dropped, i.e. understood), is clearly the strongest notion out of these three, and papers on this topic generally focus on proving this. "Chosen message" here refers to the idea that the attacker even gets to choose <em>what</em> message he will generate a verifying forgery for; with the trivial restriction that it can't be a message that the genuine signer has already signed.</p>
<p>(A minor point: you can also make this definition more precise with SUFCMA (S = "strongly"), where you insist that the finally produced signature by the attacker is not on the same message as one of the preexisting signatures. The famous problem of <strong>signature malleability</strong> experienced in ECDSA/Bitcoin relates to this, as noted by Matt Green <a href="https://blog.cryptographyengineering.com/eufcmaandsufcma/">here</a>.)</p>
<p>I believe there are even stronger notions (e.g. involving active attacks) but I haven't studied this.</p>
<p>In the next, main section of this post, I want to outline how cryptographers try to argue that both the SIDP and the Schnorr signature are secure (in the latter case, with that strongest notion of security).</p>
<h2>Why the Schnorr signature is secure</h2>
<h3>Why the SIDP is secure</h3>
<p>Here, almost by definition, we can see that only the notion of "total break" makes sense: there is no message, just an assertion of key ownership. In the context of SIDP this is sometimes called the "impersonation attack" for obvious reasons  see our reluctant scammer.</p>
<p>The justification of this is somehow elegantly and intriguingly short:</p>
<blockquote>
<p>The SIDP is secure against impersonation = The SIDP is <em>sound</em> as a ZKPOK.</p>
</blockquote>
<p>You can see that these are just two ways of saying the same thing. But what's the justification that either of them are true? Intuitively the soundness proof tries to isolate the Prover as a machine/algorithm and screw around with its sequencing, in an attempt to force it to spit out the secret that we believe it possesses. If we hypothesise an adversary \(\mathbb{A}\) who <em>doesn't</em> possess the private key to begin with, or more specifically, one that can pass the test of knowing the key for any public key we choose, we can argue that there's only one circumstance in which that's possible: <strong>if \(\mathbb{A}\) can solve the general Elliptic Curve Discrete Logarithm Problem(ECDLP) on our curve.</strong> That's intuitively <em>very</em> plausible, but can we prove it?</p>
<h3>Reduction</h3>
<p>(One of a billion variants on the web, taken from <a href="https://jcdverha.home.xs4all.nl/scijokes/6_2.html">here</a> :))</p>
<blockquote>
<pre class="joke">A mathematician and a physicist were asked the following question:
Suppose you walked by a burning house and saw a hydrant and
a hose not connected to the hydrant. What would you do?
P: I would attach the hose to the hydrant, turn on the water, and put out
the fire.
M: I would attach the hose to the hydrant, turn on the water, and put out
the fire.
Then they were asked this question:
Suppose you walked by a house and saw a hose connected to
a hydrant. What would you do?
P: I would keep walking, as there is no problem to solve.
M: I would disconnect the hose from the hydrant and set the house on fire,
reducing the problem to a previously solved form.</pre>
</blockquote>
<p></p>
<p>The general paradigm here is:</p>
<blockquote>
<p>A protocol X is "reducible to" a hardness assumption Y if a hypothetical adversary \(\mathbb{A}\) who can break X can also violate Y.</p>
</blockquote>
<p>In the concrete case of X = SIDP and Y = ECDLP we have nothing to do, since we've already done it. SIDP is intrinsically a test that's relying on ECDLP; if you can successfully impersonate (i.e. break SIDP) on any given public key \(P\) then an "Extractor" which we will now call a <strong>wrapper</strong>, acting to control the environment of \(\mathbb{A}\) and running two executions of the second half of the transcript, as already described above, will be able to extract the private key/discrete log corresponding to \(P\). So we can think of that Extractor itself as a machine/algorithm which spits out the \(x\) after being fed in the \(P\), in the simple case where our hypothetical adversary \(\mathbb{A}\) is 100% reliable. In this specific sense:</p>
<blockquote>
<p><strong>SIDP is reducible to ECDLP</strong></p>
</blockquote>
<p>However, in the real world of cryptographic research, such an analysis is woefully inadequate; because to begin with ECDLP being "hard" is a computational statement: if the group of points on the curve is only of order 101, it is totally useless since it's easy to compute all discrete logs by brute force. So, if ECDLP is "hard" on a group of size \(2^k\), let's say its hardness is measured as the probability of successfully cracking by guessing, i.e. \(2^{k}\) (here <strong>deliberately avoiding</strong> the real measure based on smarter than pure guesses, because it's detail that doesn't affect the rest). Suppose \(\mathbb{A}\) has a probability of success \(\epsilon\); what probability of success does that imply in solving ECDLP, in our "wrapper" model? Is it \(\epsilon\)?</p>
<p>No; remember the wrapper had to actually extract <strong>two</strong> successful impersonations in the form of valid responses \(s\) to challenge values \(e\). We can say that the wrapper <strong>forks</strong> \(\mathbb{A}\):</p>
<p><img alt="Fork your sigma protocol if you want fork" height="466" src="https://joinmarket.me/static/media/uploads/.thumbnails/forking.png/forking659x466.png" width="659"/></p>
<p><em>Fork your sigma protocol if you want fork</em></p>
<p><em></em></p>
<p>Crudely, the success probability is \(\epsilon^2\); both of those impersonations have to be successful, so we multiply the probabilities. (More exact: by a subtle argument we can see that the size of the challenge space being reduced by 1 for the second run of the protocol implies that the probability of success in that second run is reduced, and the correct formula is \(\epsilon^2  \frac{\epsilon}{n}\), where \(n\) is the size of the hash function output space; obviously this doesn't matter too much).</p>
<p>How does this factor into a real world decision? We have to go back to the aforementioned "reverse thinking". The reasoning is something like:</p>
<ul>
<li>We believe ECDLP is hard for our group, let's say we think you can't do better than p = \(p\) (I'll ignore running time and just use probability of success as a measure, for simplicity).</li>
<li>The above reduction implies that <em>if</em> we can break SIDP with prob \(\epsilon\), we can also break ECDLP with prob \(\simeq \epsilon^2\).</li>
<li>This reduction is thus <strong>not tight</strong>  if it's really the case that "the way to break SIDP is only to break ECDLP" then a certain hardness \(p\) only implies a hardness \(\sqrt{p}\) for SIDP, which we may not consider sufficiently improbable (remember that if \(p=2^{128}\), it means halving the number of bits: \(\sqrt{p} =2^{64}\)). See <a href="https://crypto.stackexchange.com/questions/14439/proofsbyreductionandtimesofadversaries">here</a> for a nice summary on "nontight reductions".</li>
<li>And <em>that</em> implies that if I want 128 bit security for my SIDP, I need to use 256 bits for my ECDLP (so my EC group, say). This is all handwavy but you get the pattern: these arguments are central to deciding what security parameter is used for the underlying hardness problem (here ECDLP) when it's applied in practice to a specific protocol (here SIDP).</li>
</ul>
<p>I started this subsection on "reductions" with a lame math joke; but I hope you can see how delicate this all is ... we start with something we believe to be hard, but then "solve" it with a purely hypothetical other thing (here \(\mathbb{A}\) ), and from this we imply a twoway connection (I don't say <em>equivalence</em>; it's not quite that) that we use to make concrete decisions about security. Koblitz (he of the 'k' in secp256k1) had some interesting thoughts about 'reductionist' security arguments in Section 2.2 and elsewhere in <a href="https://cr.yp.to/bib/2004/koblitz.pdf">this</a> paper. More from that later.</p>
<p>So we have sketched out how to think about "proving our particular SIDP instance is/isn't secure based on the intractability of ECDLP in the underlying group"; but that's only 2 stacks in our jenga tower; we need MOAR!</p>
<h2>From SIDP to Schnorr signature</h2>
<p>So putting together a couple of ideas from previous sections, I hope it makes sense to you now that we want to prove that:</p>
<blockquote>
<p>"the (EC) Schnorr signature has existential unforgeability against chosen message attack (EUFCMA) <strong>if</strong> the Schnorr Identity Protocol is secure against impersonation attacks."</p>
</blockquote>
<p>with the understanding that, if we succeed in doing so, we have proven also:</p>
<blockquote>
<p>"the (EC) Schnorr signature has existential unforgeability against chosen message attack (EUFCMA) <strong>if</strong> the Elliptic Curve discrete logarithm problem is hard in our chosen EC group."</p>
</blockquote>
<p>with the substantial caveat, as per the previous section, that the reduction involved in making this statement is not tight.</p>
<p>(there is another caveat though  see the next subsection, <em>The Random Oracle Model</em>).</p>
<p>This second (third?) phase is much less obvious and indeed it can be approached in more than one way. <a href="https://crypto.stanford.edu/~dabo/cryptobook/">BonehShoup</a> deals with it in a lot more detail; I'll use this as an outline but dumb it down a fair bit. There is a simpler description <a href="http://web.stanford.edu/class/cs259c/lectures/schnorr.pdf">here</a>.</p>
<p>The "CMA" part of "EUFCMA" implies that our adversary \(\mathbb{A}\), who we are now going to posit has the magical ability to forge signatures (so it's the black cube of our preamble), should be able to request signatures on an arbitrarily chosen set of messages \(m_i\), with \(i\) running from 1 to some defined number \(S\). But we must also allow him to make queries to the hash function, which we idealise as a machine called a "random oracle". Brief notes on that before continuing:</p>
<h3>Aside: The Random Oracle Model</h3>
<p>Briefly described <a href="https://en.wikipedia.org/wiki/Random_oracle">here</a> . It's a simple but powerful idea: we basically idealise how we want a cryptographic hash function \(f\) to behave. We imagine an output space for \(f\) of size \(C\). For any given input \(x\) from a predefined input space of one or more inputs, we will get a deterministic output \(y\), but it should be unpredictable, so we imagine that the function is <em>randomly</em> deterministic. Not a contradiction  the idea is only that there is no <strong>public </strong>law or structure that allows the prediction of the output without actually passing it through the function \(f\)<em></em>. The randomness should be uniform.<em> </em></p>
<p>In using this in a security proof, we encounter only one problem: we will usually want to model \(f\) by drawing its output \(y\) from a uniformly random distribution (you'll see lines like \(y \stackrel{$}{\leftarrow} \mathbb{Z}_N\) in papers, indicating \(y\) is set randomly). But in doing this, we have set the value of the output for that input \(x\) permanently, so if we call \(f\) again on the same \(x\), whether by design or accident, we <em>must</em> again return the same "random" \(y\).</p>
<p>We also find sometimes that in the nature of the security game we are playing, one "wrapper" algorithm wants to "cheat" another, wrapped algorithm, by using some hidden logic to decide the "random" \(y\) at a particular \(x\). This <em>can</em> be fine, because to the "inner" algorithm it can look entirely random. In this case we sometimes say we are "<strong>patching the value of the RO at \(x\) to \(y\)"</strong> to indicate that this artificial event has occurred; as already mentioned, it's essential to remember this output and respond with it again, if a query at \(x\) is repeated.</p>
<p>Finally, this "perfectly random" behaviour is very idealised. Not all cryptographic protocols involving hash functions require this behaviour, but those that do are said to be "secure in the random oracle model (ROM)" or similar.</p>
<h3>Wrapping A with B</h3>
<p><img alt="B tries to win the impersonation game against C, by wrapping the signature forger A" height="413" src="https://joinmarket.me/static/media/uploads/.thumbnails/EUFCMA1.png/EUFCMA1584x413.png" width="584"/></p>
<p>So we now wrap \(\mathbb{A}\) with \(\mathbb{B}\). And \(\mathbb{B}\)'s job will be to succeed at winning the SIDP "game" against a challenger \(\mathbb{C}\) .</p>
<p>Now \(\mathbb{A}\) is allowed \(S\) signing queries; given his messages \(m_i\), we can use \(S\) eavesdropped conversations \(R, e, s\) from the actual signer (or equivalently, just forge transcripts  see "zero knowledgeness" of the Schnorr signature), and for each, \(\mathbb{B}\) can patch up the RO to make these transcripts fit \(\mathbb{A}\)'s requested messages; just do \(H(m_iR_i)=e_i\). Notice that this part of the process represents \(S\) queries to the random oracle.</p>
<p>Observe that \(\mathbb{B}\) is our real "attacker" here: he's the one trying to fool/attack \(\mathbb{C}\) 's identification algorithm; he's just using \(\mathbb{A}\) as a black box (or cube, as we say). We can say \(\mathbb{A}\) is a "subprotocol" used by \(\mathbb{B}\).</p>
<p>It's all getting a bit complicated, but by now you should probably have a vague intuition that this will work, although of course not reliably, and as a function of the probability of \(\mathbb{A}\) being able to forge signatures of course (we'll again call this \(\epsilon\)).</p>
<h3>Toy version: \(\epsilon = 1\)</h3>
<p>To aid understanding, imagine the simplest possible case, when \(\mathbb{A}\) works flawlessly. The key \(P\) is given to him and he chooses a random \(k, R =kG\), and also chooses his message \(m\) as is his right in this scenario. The "CMA" part of EUFCMA is irrelevant here, since \(\mathbb{A}\) can just forge immediately without signature queries:</p>
<ul>
<li>\(\mathbb{A}\) asks for the value of \(H(mR)\), by passing across \(m,R\) to \(\mathbb{B}\).</li>
<li>\(\mathbb{B}\) receives this query and passes \(R\) as the first message in SIDP to \(\mathbb{C}\) .</li>
<li>\(\mathbb{C}\) responds with a completely random challenge value \(e\).</li>
<li>\(\mathbb{B}\) "patches" the RO with \(e\) as the output for input \(m, R\), and returns \(e\) to \(\mathbb{A}\) .</li>
<li>\(\mathbb{A}\) takes \(e\) as \(H(mR)\), and provides a valid \(s\) as signature.</li>
<li>\(\mathbb{B}\) passes \(s\) through to \(\mathbb{C}\) , who verifies \(sG = R + eP\); identification passed.</li>
</ul>
<p>You can see that nothing here is new except the random oracle patching, which is trivially nonproblematic as we make only one RO query, so there can't be a conflict. The probability of successful impersonation is 1.</p>
<p>Note that this implies the probability of successfully breaking ECDLP is also \(\simeq 1\). We just use a secondlayer wrapper around \(\mathbb{B}\), and fork its execution after the provision of \(R\), providing two separate challenges and thus in each run getting two separate \(s\) values and solving for \(x\), the private key/discrete log as has already been explained.</p>
<p>Why \(\simeq\)? As noted on the SIDP to ECDLP reduction above, there is a tiny probability of a reused challenge value which must be factored out, but it's of course negligible in practice.</p>
<p>If we assert that the ECDLP is not trivially broken in reasonable time, we must also assert that such a powerful \(\mathbb{A}\) does not exist, given similarly limited time (well; <em>in the random oracle model</em>, of course...).</p>
<h3>Full CMA case, \(\epsilon << 1\)</h3>
<p>Now we give \(\mathbb{A}\) the opportunity to make \(S\) signing queries (as already mentioned, this is what we mean by an "adaptive chosen message attack"). The sequence of events will be a little longer than the previous subsection, but we must think it through to get a sense of the "tightness of the reduction" as already discussed.</p>
<p>The setup is largely as before: \(P\) is given. There will be \(h\) RO queries allowed (additional to the implicit ones in the signing queries).</p>
<ul>
<li>For any signing query from \(\mathbb{A}\), as we covered in "Wrapping A with B", a valid response can be generated by patching the RO (or using real transcripts). We'll have to account for the possibility of a conflict between RO queries (addressed below), but it's a minor detail.</li>
<li>Notice that as per the toy example previously, during \(\mathbb{A}\)'s forgery process, his only interaction with his wrapper \(\mathbb{B}\) is to request a hash value \(H(mR)\). So it's important to understand that, first because of the probabilistic nature of the forgery (\(\epsilon << 1\)), and second because \(\mathbb{A}\)'s algorithm is unknown, <strong>\(\mathbb{B}\) does not know which hash function query (and therefore which RO response) will correspond to a successful forgery. </strong>This isn't just important to the logic of the game; as we'll see, it's a critical limitation of the security result we arrive at.</li>
<li>So to address the above, \(\mathbb{B}\) has to make a decision upfront: which query should I use as the basis of my impersonation attempt with \(\mathbb{C}\)? He chooses an index \(\omega\ \in 1..h\).</li>
<li>There will be a total of \(S+h+1\) queries to the random oracle, at most (the +1 is a technical detail I'll ignore here). We discussed in the first bullet point that if there is a repeated \(m, R\) pair in one of the \(S\) signing queries, it causes a "conflict" on the RO output. In the very most pessimistic scenario, the probability of this causing our algorithm to fail can be no more than \(\frac{S+h+1}{n}\) for each individual signing query, and \(\frac{S(S+h+1)}{n}\) for all of them (as before we use \(n\) for the size of the output space of the hash function).</li>
<li>So \(\mathbb{B}\) will <strong>fork</strong> \(\mathbb{A}\)'s execution, just as for the SIDP \(\rightarrow\) ECDLP reduction, <strong>at index \(\omega\)</strong>, without knowing in advance whether \(\omega\) is indeed the index at the which the hash query corresponds to \(\mathbb{A}\)'s final output forgery. There's a \(1/h\) chance of this guess being correct. So the "partial success probability", if you will, for this first phase, is \(\epsilon/h\), rather than purely \(\epsilon\), as we had for the SIDP case.</li>
<li>In order to extract \(x\), though, we need that the execution <em>after</em> the fork, with the new challenge value, at that same index \(\omega\), also outputs a valid forgery. What's the probability of both succeeding together? Intuitively it's of the order of \(\epsilon^2\) as for the SIDP case, but clearly the factor \(1/h\), based on accounting for the guessing of the index \(\omega\), complicates things, and it turns out that the statistical argument is rather subtle; you apply what has been called the <strong>Forking Lemma</strong>, described on <a href="https://en.wikipedia.org/wiki/Forking_lemma">Wikipedia</a> and with the clearest statement and proof in <a href="https://cseweb.ucsd.edu/~mihir/papers/multisignaturesccs.pdf">this</a> paper of BellareNeven '06. The formula for the success probability of \(\mathbb{B}\) turns out to be:</li>
</ul>
<blockquote>
<p>\(\epsilon_{\mathbb{B}} = \epsilon\left(\frac{\epsilon}{h}  \frac{1}{n}\right)\)</p>
</blockquote>
<ul>
<li><a href="https://crypto.stanford.edu/~dabo/cryptobook/">BonehShoup</a> in Section 19.2 bundle this all together (with significantly more rigorous arguments!) into a formula taking account of the Forking Lemma, the accounting for collisions in the signing queries, to produce the more detailed statement, where \(\epsilon\) on the left here refers to the probability of success of \(\mathbb{B}\), and "DLADv" on the right refers to the probability of success in solving the discrete log. The square root term of course corresponds to the "reduction" from Schnorr sig. to ECDLP being roughly a square:</li>
</ul>
<blockquote>
<p>\(\epsilon \le \frac{S(S+h+1)}{n} + \frac{h+1}{n} + \sqrt{(h+1)\ \times \ \textrm{DLAdv}}\)</p>
</blockquote>
<p>So in summary: we see that analysing the full CMA case in detail is pretty complicated, but by far the biggest take away should be: <strong>The security reduction for Schnorr sig to ECDLP has the same \(\epsilon^2\) dependency, but is nevertheless far less tight, because the success probability is also reduced by a factor \(\simeq h\) due to having to guess which RO query corresponds to the successful forgery.</strong></p>
<p>(<em>Minor clarification: basically ignoring the first two terms on the RHS of the preceding as "minor corrections", you can see that DLAdv is very roughly \(\epsilon^2/h\)</em>).</p>
<p>The above bolded caveat is, arguably, very practically important, not just a matter of theory  because querying a hash function is something that it's very easy for an attacker to do. If the reduction loses \(h\) in tightness, and the attacker is allowed \(2^{60}\) hash function queries (note  they can be offline), then we (crudely!) need 60 bits more of security in our underlying cryptographic hardness problem (here ECDLP); at least, <em>if</em> we are basing our security model on the above argument.</p>
<p>Although I haven't studied it, <a href="https://eprint.iacr.org/2012/029">the 2012 paper by Yannick Seurin</a> makes an argument (as far as I understand) that we cannot do better than this, in the random oracle model, i.e. the factor of \(h\) cannot be removed from this security reduction by some better kind of argument.</p>
<p></p>
<h2>Summary  is Schnorr secure?</h2>
<p>For all that this technical discussion has exposed the nontrivial guts of this machine, it's still true that the argument provides some pretty nice guarantees. We can say something like "Schnorr is secure if:"</p>
<ul>
<li>The hash function behaves to all intents and purposes like an ideal random oracle as discussed</li>
<li>The ECDLP on our chosen curve (secp256k1 in Bitcoin) is hard to the extent we reasonably expect, given the size of the curve and any other features it has (in secp256k1, we hope, no features at all!)</li>
</ul>
<p>This naturally raises the question "well, but how hard <em>is</em> the Elliptic Curve discrete logarithm problem, on secp256k1?" Nobody really knows; there are known, standard ways of attacking it, which are better than brute force unintelligent search, but their "advantage" is a roughly known quantity (see e.g. <a href="https://en.wikipedia.org/wiki/Pollard%27s_rho_algorithm">Pollard's rho</a>). What there isn't, is some kind of proof "we know that \(\nexists\) algorithm solving ECDLP on (insert curve) faster than \(X\)".</p>
<p>Not only don't we know this, but it's even rather difficult to make statements about analogies. I recently raised the point on #bitcoinwizards (freenode) that I thought there must be a relationship between problems like RSA/factoring and discrete log finding on prime order curves, prompting a couple of interesting responses, agreeing that indirect evidence points to the two hardness problems being to some extent or other connected. Madars Virza kindly pointed out a <a href="https://wstein.org/projects/john_gregg_thesis.pdf#page=43">document</a> that details some ideas about the connection (obviously this is some pretty highbrow mathematics, but some may be interested to investigate further).</p>
<h2>What about ECDSA?</h2>
<p>ECDSA (and more specifically, DSA) were inspired by Schnorr, but have design decisions embedded in them that make them <em>very</em> different when it comes to security analysis. ECDSA looks like this:</p>
<blockquote>
<p>\(s = k^{1}\left(H(m) + rx\right), \quad r=R.x, \ R = kG\)</p>
</blockquote>
<p>The first problem with trying to analyse this is that it doesn't conform to the threemovesigmaprotocolidentificationschemeconvertstosignatureschemeviaFiatShamirtransform. Why? Because the hash value is \(H(m)\) and doesn't include the commitment to the nonce, \(R\). This means that the standard "attack" on Schnorr, via rewinding and resetting the random oracle doesn't work. This doesn't of course mean, that it's insecure  there's another kind of "fixing" of the nonce, in the setting of\(R.x\). This latter "conversion function" kind of a random function, but really not much like a hash function; it's trivially "semiinvertible" in as much as given an output xcoordinate one can easily extract the two possible input Rvalues.</p>
<p>Some serious analysis has been done on this, for the obvious reason that (EC)DSA is <strong>very widely used in practice.</strong> There is work by <a href="https://www.iacr.org/archive/pkc2003/25670309/25670309.pdf">Vaudenay</a> and <a href="https://www.cambridge.org/core/books/advancesinellipticcurvecryptography/ontheprovablesecurityofecdsa/69827A20CC94C54BBCBC8A51DBAF075A">Brown</a> (actually a few papers but I think most behind academic paywalls) and most recently <a href="https://dl.acm.org/citation.cfm?doid=2976749.2978413">Fersch et al</a>. Fersch gave a talk on this work <a href="https://www.youtube.com/watch?v=5aUPBT4Rdr8">here</a> .</p>
<p>The general consensus seems to be "it's very likely secure  but attempting to get a remotely "clean" security reduction is very difficult compared to Schnorr".</p>
<p>But wait; before we trail off with an inaudible mumble of "well, not really sure..."  there's a crucial logical implication you may not have noticed. Very obviously, ECDSA is not secure if ECDLP is not secure (because you just get the private key; game over for any signature scheme). Meanwhile, in the long argument above we <strong>reduced</strong> Schnorr to ECDLP. This means:</p>
<blockquote>
<p><strong>If ECDSA is secure, Schnorr is secure, but we have no security reduction to indicate the contrary.</strong></p>
</blockquote>
<p>The aforementioned Koblitz paper tells an interesting historical anecdote about all this, when the new DSA proposal was first put forth in '92 (emphasis mine):</p>
<blockquote>
<p>"At the time, the proposed standard — which soon after became the first digital signature algorithm ever approved by the industrial standards bodies — encountered stiff opposition, especially from advocates of RSA signatures and from people who mistrusted the NSA’s motives. Some of the leading cryptographers of the day tried hard to find weaknesses in the NIST proposal. A summary of the most important objections and the responses to them was published in the Crypto’92 proceedings[17]. The opposition was unable to find any significant defects in the system. <span style="textdecoration: underline;">In retrospect, it is amazing that none of the DSA opponents noticed that when the Schnorr signature was modified, the equivalence with discrete logarithms was lost.</span>"</p>
</blockquote>
<h2>More exotic constructions</h2>
<p>In a future blog post, I hope to extend this discussion to other constructions, which are based on Schnorr in some way or other, in particular:</p>
<ul>
<li>The AOS ring signature</li>
<li>The FujisakiSuzuki, and the cryptonote ringsig</li>
<li>the LiuWeiWong, and the Monero MLSAG (via Adam Back) ringsig</li>
<li>The MuSig multisignature</li>
</ul>
<p>While these are all quite complicated (to say the least!), so no guarantee of covering all that, the security arguments follow similar lines to the discussion in this post. Of course ring signatures have their own unique features and foibles, so I will hopefully cover that a bit, as well as the security question.</p>
<p></p>Finessing commitments20190129T20:11:46+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/finessingcommitments/<h2>Introduction</h2>
<p>This post was mostly prompted by a long series of discussions had online and in person with many people, including in particular Adam Back and Tim Ruffing (but lots of others!)  and certainly not restricted to discussions I took part in  about the tradeoffs in a version of Bitcoin that does actually use Confidential Transactions.</p>
<p>The topic that keeps recurring is: exactly what level of safety against <em>hidden inflation</em> does CT offer, in principle and in practice; this is closely related to what level of privacy it offers, too, but the hidden inflation is what gets people thinking, first and foremost.</p>
<p>My goal here is to explain to people who are not completely up to speed with what causes this discussion; to explain, in detail but without assuming <em>too</em> much preknowledge, what the heart of the tradeoff is, and in the last couple of sections, how we might get around it. How we might "have our cake and eat it" so to speak.</p>
<p><em>You'll find a lot of the ideas in first three sections of this blog post, although not all of them, in my write up on <a href="https://github.com/AdamISZ/from0k2bp">Bulletproofs</a> section 3, and I also went over the basics in the early part of my London talk on the <a href="https://www.youtube.com/watch?v=mLZ7qVwKalE">Schnorr signature</a> (since they're not visible, you'd want to see the <a href="https://joinmarket.me/static/schnorrplus.pdf">slides</a> for that talk if you watch it).</em></p>
<p><em>You should have <strong>some</strong> general idea about what Confidential Transactions is, in order to understand the second half  in particular you should understand that amounts are hidden under Pedersen commitments, although we'll go through some of it here in the early sections.</em></p>
<h2>Commitments  the basic ideas</h2>
<p>A commitment fixes a value in advance, without revealing it. Think of flipping a coin and covering it with your hand as part of a gamble or game. Covering it with your hand means it's invisible. The visibility of your hand not moving, and pressed against a surface over the coin, on the other hand, ensures you can't cheat, because you can't flip the coin under your hand. This is the physical representation of the two ideas of a commitment; everything that cryptographers call a commitment has those two properties, but defended by very strong mathematical "guarantees" (more on those quote marks shortly!), instead of by a crappy physical analogue like a presseddown hand:</p>
<ul>
<li><code>Hiding</code>  nobody but the committer can see or infer the actual value being committed</li>
<li><code>Binding</code>  the committer can't change the value after the commitment is published.</li>
</ul>
<p>The most "vanilla" way to implement this primitive is to use a cryptographically secure hash function, say \(\mathbb{H}\). You'd make up a random number \(r\) and combine it with your secret value \(x\), and the commitment would just be something like \(\mathbb{H}(xr)\), where  indicates just concatenating the data together.</p>
<p>For background, commitments are usually defined in the literature as a tuple of three algorithms: Setup, Commit and Verify/Open, where the last one needs the committing party to give what's called the "opening" of the commitment as input. It should be clear what is meant when I say that the "opening" of the above commitment is just literally the two values \(x\) and \(r\). (It's like taking your hand off the coin in our analogue).</p>
<p>Because (cryptographically secure) hashfunctions are collisionresistant (avoiding technical definitions for brevity here), you can't open a commitment you already made to \(x\), with a different value \(x'\), i.e. as committer you can't cheat, because even though you're free to make your cheating opening operation with any \(r'\) you like (it wasn't published in advance), you just can't find any combination \(x',r'\) such that \(\mathbb{H}(xr)=\mathbb{H}(x'r')\). That's why you need a proper, strong hash function  to make that computationally infeasible.</p>
<h2>Homomorphic commitments</h2>
<p>The strict definition of homomorphism requires going into group theory a bit, which isn't needed here, but basically a homomorphism should be thought of as a function that takes you from one group into another while keeping the group operation intact (so it's closely related to symmetry). A toy example would be the homomorphism from the group of integers under addition (\(\mathbb{Z},+)\) to the group of integers modulo two \(\mathbb{Z}_2\) (a very simple group!: two members: 0, 1; each is selfinverse; 0+1=1+0 = 1, 0+0=1+1 =0). What is that homomorphism? It's just a function that returns 0 if the input integer is even, and 1 if the input integer is odd. We lost everything about the integers except their evenness/oddness but we kept the group operation. Call that function \(f()\) and we clearly have \(f(a+b)=f(a)+f(b)\) because even + odd = odd, odd + even = odd, even + even = odd + odd = even.</p>
<p>But homomorphisms need not collapse a larger group into a smaller one, the "throw away everything except X" can still conceivably mean throwing away nothing, i.e. mapping from one group to another with the same order  and the homomorphism that's relevant to this discussion fits that description: it translates members of the group of integers modulo \(N\) under addition, to the group of elliptic curve points of order \(N\), under elliptic curve point addition: \(a \in \mathbb{Z}_N \ ; a \rightarrow aG\), where \(G\) is the socalled generator point of the elliptic curve (exactly which point on the curve is taken for this role is irrelevant, but the definition has to include one, and we call it \(G\)), and implicit in the notation \(aG\) is the fact that it means a <em>scalar multiplication</em> of \(G\) by \(a\), i.e. \(G\) added to itself \(a\) times.</p>
<p>In Bitcoin, the curve in question is secp256k1, but that's not important here (except perhaps the fact that \(N\) is prime, meaning both groups are isomorphic to the cyclic group of order \(N\)).</p>
<p>So this is all getting technical, but all it really boils down to is \((a+b)G = aG + bG\) is an equation that holds, here.</p>
<p>What about commitments? We can treat the above homomorphism as <em>similar </em>to a cryptographic hash function, in that we can <em>assume</em> that it's not possible to derive the integer \(a\) given only the curve point \(aG\)  this assumes that the "elliptic curve discrete logarithm problem" is hard (ECDLP for short; also, that's kind of the definition of that problem).</p>
<p>In making that assumption, we can go ahead and apply the same paradigm: take a random \(r\) for hiding, then take a second generator point \(H\) and write our commitment as \(xG + rH\) (addition not concatenation; publishing the two curve points separately would defeat the purpose; \(r\) is supposed to be helping to hide \(x\)!).</p>
<p>And we note how the homomorphism between an integer and the scalar multiple of \(G\) by that integer carries over to this composite case of the sum of two points: \(x_{1}G + r_{1}H + x_{2}G + r_{2}H = (x_{1}+x_{2})G + (r_{1}+r_{2})H\).</p>
<p>So it means the commitments, which we can denote \(C(x, r)\) for brevity, <strong>are homomorphic</strong> too. The sum of two commitments is equal to the commitment to the sum (in this sentence, notice how "sum" refers to the sum of two integers; but the homomorphism allows that to "carry over" to "sum" as in elliptic curve point addition). Algebraically we can condense it to: \(C(x_1,r_1)+C(x_2,r_2) = C(x_{1}+x_{2},r_{1}+r_{2})\).</p>
<p>This specific form of commitment is known as the <strong>Pedersen commitment</strong>. It is most commonly referred to in the finite field, non elliptic curve form (where \(C=g^{x}h^{r}\) rather than \(C=xG + rH\)), but it's the same thing.</p>
<p>And to reemphasise what at this point should be obvious  <strong>none</strong> of the above applies to commitments built with cryptographic hash functions.</p>
<p></p>
<h2>Imperfect commitments</h2>
<p>The first imperfection in the idea above of the Pedersen commitment: the second generator point \(H\). In practice it has been calculated in a <a href="https://en.wikipedia.org/wiki/Nothing_up_my_sleeve_number">NUMS</a> way, using some kind of hash of the defined generator point \(G\). The idea is that if this hash function \(\mathbb{H}\) is secure as described above, \(H\) cannot be reverseengineered such that its discrete log (that's to say, \(\gamma\) s.t. \(H=\gamma G\)). And while this seems like a sidenote, we can use this to lead in to the subtleties which are the main topic of this blog post.</p>
<p>Consider (and this is an excellent exercise for those <em>somewhat </em>familiar with basic elliptic curve operations as used in Bitcoin and similar, but not yet seasoned in it): if you secretly knew that \(\gamma\) and no one else did, and Pedersen commitments were being used, how could you use this knowledge to gain advantage?</p>
<p>(Spoiler space)</p>
<p>.</p>
<p>.</p>
<p>.</p>
<p>.</p>
<p>The answer is that the a commitment would lose its <strong>binding</strong> property. Remember: the binding property is the defence the receiver/verifier of the commitment has against the creator/committer. So you would be able to make a commitment randomly  just take any random "private key", call it \(y\), and publish its "public key" point \(yG\) as \(C\). Then when asked to verify later, you could pretend that you had bound this commitment to any \(x\) as follows: Take your newly chosen \(x\), and calculate what \(r\) has to be for that to work:</p>
<p>\(xG + rH = C\)<br/>\(xG + r\gamma G = yG\)<br/>\(\therefore\)<br/>\(r = \gamma^{1}(yx)\)</p>
<p>There are two thoughts that may spring out of this:</p>
<ul>
<li>A Pedersen commitment then, is only sound in as much as the relative discrete log between \(H\) and \(G\) is unknown. And as is often discussed, the aforementioned ECDLP is almost certainly broken by a sufficiently powerful quantum computer (although there might conceivably other ways to compute a discrete log quickly, as yet unknown to mathematicians). We say that Pedersen commitments are only <span style="textdecoration: underline;">computationally binding<strong></strong></span>  this means, only binding inasmuch as breaking their binding requires an unrealistic amount of computational power  to an adversary with infinite computing power, they are <strong>not</strong> binding, at all.</li>
<li>You may think  what about hiding? Surely knowing this "secret unlocking key" \(\gamma\) can break that as well. The answer is an emphatic <strong>no</strong>  but what might be really surprising is: <em>the answer is no in the case of hiding for exactly the same reason that the answer is yes, in the case of binding!</em></li>
</ul>
<p>I'm first going to explain that formally, but then we'll take a step back, and use a picture and perhaps some examples to flesh out what's really going on here, because it's the heart of the story we're telling.</p>
<p>The reason hiding is not lost is because an adversary on the receiving/verifying side who wants to "sneak peek" inside \(C\) to find out the real \(x\) faces an insurmountable problem: <span style="textdecoration: underline;">there isn't only one answer!</span>. If the answer is \(x\) then the \(r\) value is, as already explained, \(\gamma^{1}(yx)\). Remember, a computationally unbound attacker can find those discrete logs  can get \(y\) from \(C\), and can get \(\gamma\) from \(H\). So <em>any</em> \(x\) will have a corresponding \(r\). And of course you can flip it backwards  if he fixes a particular \(r\) he can get the corresponding \(x\). Where is this slipperiness coming from? What's it's fundamental cause?</p>
<p>Think about functions. They take inputs to outputs of course, and we generalise by talking about input spaces and output spaces (by "space" here I mean something like "the set of all..."). The technical terms used are domain/range and sometimes preimage space and image space. The input space for the private>public key "function" in Bitcoin is of course the set of all integers modulo \(N\). And then output space is the set of all public keys, which is the set of all points on the curve secp256k1. Here we have what's called a "oneone" mapping (technically it's both "oneone" and also "onto", since all points in the output space are mapped to). But only "nice" functions are like that, not all are. Some, in particular, have <strong>more than one input corresponding to the same output</strong>.</p>
<p>And that's exactly what's happening with the Pedersen commitment; moreover, there's a very concrete reason <em>why </em>the Pedersen commitment has that property of having multiple inputs for the same output  it logically has to! Consider this diagram:</p>
<p><img alt="Input space larger than output space" height="487" src="https://joinmarket.me/static/media/uploads/.thumbnails/InputOutput1.png/InputOutput1689x487.png" width="689"/></p>
<p>By the <a href="https://en.wikipedia.org/wiki/Pigeonhole_principle">pigeonhole principle</a>, because there are more inputs than outputs, it's impossible for it <em>not</em> to be the case that at least some outputs have more than one corresponding input  they wouldn't all "fit" otherwise. And it's for this reason that <span style="textdecoration: underline;">a commitment scheme where the input space is larger than the output space can have perfect hiding.</span> (notice "can", not "will"  for the hiding to be *perfect* we need to be leaking zero information about which element of the input space is being hidden; that needs it to be the case that we can demonstrate that *any* element of some very large subset of the input space is part of a valid opening of a given commitment \(C\); that's true here in Pedersen, but certainly not for some other commitment schemes).</p>
<p>And for the exact same reason, the binding is only computational  there are bound to be at least some outputs with more than one input, and so if nothing else, by exhaustive search, the computationally unbounded attacker can simply search and find the other input corresponding to the same output (as per the diagram, if the attacker had the value \(k\) as commitment, he could find \(x_2, r_2\) even if the commitment was originally to \(x_1, r_1\). At least that'll be true for <em>some</em> outputs (in Pedersen, for every output, in fact).</p>
<p>So the Pedersen commitment falls neatly into this category; it has an input space of 512 bits if the message \(x\) and randomness \(r\) are both the same size as the group elements and an output space of 256 bits (almost exactly); the outputs \(C\) are the points on the secp256k1 curve.</p>
<p>(You'll notice how technically "exhaustive search" may not be the actual process  there can be shortcuts depending on how the commitment is structured; in the case of Pedersen, because it hinges on the unknown discrete log \(\gamma\), the attacker can leverage that  he can use that knowledge to find these "other inputs" directly instead of by exhaustive search).</p>
<p>What if the input space is <em>smaller </em>than the output space? It takes a moment of reflection to realise that this idea doesn't make sense. A function has a single output by definition (note that doesn't mean the output can't be multiple "things", e.g. it could be 4 integers or 6 curve points or 5 triangles, whatever  but each of them is a single output). So a function with 10 inputs can't have more than 10 outputs. Which means we have one case remaining:</p>
<p>What if the input space is <em>the same size as <strong></strong></em>the output space?</p>
<p><img alt="Input and output space equal size" height="487" src="https://joinmarket.me/static/media/uploads/.thumbnails/InputOutput2.png/InputOutput2688x487.png" width="688"/></p>
<p>In this case we must have a oneone mapping  again by the pigeonhole principle (Remember, we are defining the output space as the space of actual possible outputs, not some larger set; this will usually require justification  you can justify it here by first considering the second commitment point \(C_2=rG\)  note that the lines are horizontal for a reason!). And by the same reasoning as above, but in reverse, we see that this gives <strong>perfect</strong> binding, and <strong>at best computational (imperfect) hiding.</strong></p>
<p>What's neat about this reasoning is that none of it is specific to anything elliptic curve, or discrete log related, or anything  it applies to <em>any</em> commitment scheme, including the hashbased one we introduced right at the start. The only difference between the Pedersen and hashbased case is because of the messiness mathematically of hash functions, we can't really talk about perfection; it's only the limitations, the negative parts of the above logic, that are the same:</p>
<p>If your output space is the space of SHA256 outputs, then it's 256 bits. Now according to specification, that hash function can take extremely large input (I forget exactly, but vastly larger than 256 bits), which means it is in the first category  its input space is vastly larger than its output space, so it <strong>cannot</strong> be perfectly binding. But that <em>doesn't</em> mean that it's perfectly hiding, unfortunately  that would require that a given output leaks precisely zero information about the corresponding input. But it's certainly not the case that we have some theorem about SHA256 that ensures that every input is equiprobable, given an arbitrary output. So at <em>best</em> we have computational hiding, and that's based on the idea tha the hash function is well designed. See <a href="https://en.wikipedia.org/wiki/Cryptographic_hash_function">Wikipedia</a> for a reminder on the key properties of cryptographic hash functions. These properties are also what provides the argument for at least a computational binding. But again, it's certainly not perfectly either hiding <em>or</em> binding.</p>
<p></p>
<p>So let's summarize the key insight:</p>
<p><strong>It's LOGICALLY impossible for a commitment scheme to be both perfectly hiding and perfectly binding, no matter what algorithm or mathematical architecture is used to construct it.</strong></p>
<p>Why "logically"? Because we've demonstrated the two ideas are fundamental contradictions of each other; it is only confusion to think you can get both at the same time. Another way to say it (slightly more dynamic description):</p>
<p><strong>A commitment scheme which has been constructed to have perfect binding will at BEST achieve computational hiding, while a scheme constructed to achieve perfect hiding will at BEST achieve computational binding.</strong></p>
<p>Here we're emphasizing that these are the limits, only achieved by well designed algorithms; a badly designed or notfitforpurpose commitment scheme may not be perfect in <em>either</em> sense, and for example may not even manage to be computationally hiding, e.g. an adversary may very feasibly be able to break the hiding property without excessive computational resources. This is just a description of the <em>best</em> we can do.</p>
<h2>From hidden to bound</h2>
<p>We'll get into the Bitcoinrelated application shortly, but for now note that is not unreasonable to prefer binding over hiding in the tradeoff. Since clearly Pedersen doesn't fit there, what does?</p>
<p>Let's start with an almostobvious idea: suppose I want to commit to a value \(x\) and have it be perfectly binding. Can I just use \(C=xG\) as the commitment?</p>
<p>If you've been following along, you'll probably be a little uncertain, because .. the "hiding" part doesn't seem to have been implemented. You're right to be uncertain, because the answer is really "formally no, but it kinda depends".</p>
<p>There are two scenarios: if the set of values you might commit to is restricted in some way, e.g. a number between 1 and \(2^{25}\) then the lack of hiding makes the commitment a complete failure, because a computer could just find it by brute force guessing. And if your \(x\) was a random value in the entire range \(2^{256}\) of elements of the group  this kind of construction <em>is</em> sometimes used as a commitment, but it doesn't count as a proper, generic commitment <em>scheme</em>, because it doesn't have even computational hiding in the general case; if I <em>think</em> I know what your \(x\) is (or know the range of it), I can just check if I'm right; there is no blinding value \(r\) to prevent that.</p>
<p>This naturally leads us to the <a href="https://en.wikipedia.org/wiki/ElGamal_encryption">ElGamal encryption scheme</a>, repurposed as a commitment scheme (this can be done with any cryptosystem, by the way):</p>
<p>Take our usual suspects \((x, r)\) and construct <strong>two </strong>elliptic curve points: \((xG+rH, rG)\). This is the ElGamal commitment (with all notation as for Pedersen). Wait, I hear you cry  you're just regurgitating the Pedersen commitment, but adding \(rG\)? What does that mean? Well, we're taking the slightly broken idea above and applying it <em>in conjunction with</em> the idea of the Pedersen commitment. We "commit" to the value \(r\) using \(rG\), and that's OK specifically because \(r\) is a random number in the range (a bit like a "nonce") used just for this commitment, so there is no guessing it outside the standard brute force or breaking ECDLP; by doing so we've increased the <strong>output space</strong> from Pedersen's set of single curve points to the Cartesian product of 2 sets of curve points. And we by doing so arrive at the second of the two scenarios described in the two diagrams above; now, for each input tuple \((x, r)\), there is an output tuple \((C_1,C_2) = (xG+rH,rG)\)  guaranteed distinct because the mapping from \(r\) to \(rG\) is oneone  so the mapping is oneone and is perfectly binding. More simply: the task of the adversary who wants to break the commmitment by opening it to a different value than originally chosen is now impossible: for \(rG\) there is precisely one and only one \(r\), and once \(r\) is set, there is only one \(x\): it's the discrete log of \(C_1 rH\) which is now uniquely defined, once \(r\) is.</p>
<p>And, following our insights above, it is now decidely unsurprising to learn that the described ElGamal commitment is only computationally hiding: because \(rG\) is published as a separate curve point \(C_2\) and not folded into the single curve point as with Pedersen, an attacker with the ability to solve the discrete log problem can extract, from that, \(r\) and then follow up by extracting from \(C_1  rH=xG\), the supposedly hidden committed value \(x\).</p>
<p>But let's be clear: it <em>is </em>computationally hiding, unlike our toy "quasicommitment" \(xG\) which fails at that task (imagine committing to the value "2"). And that can be expressed formally with what's called a "reduction proof"; a rather weird but also very clever concept often used in cryptography:</p>
<pre>The ElGamal commitment is hiding if the DDH problem is hard,<br/>because an adversary who can violate the hiding property of the ElGamal<br/>commitment can use that algorithm to solve the DDH problem.</pre>
<p>DDH refers to the <a href="https://en.wikipedia.org/wiki/Decisional_Diffie%E2%80%93Hellman_assumption">Decisional Diffie Hellman problem</a>  in words, it's that you can't distinguish \(abG\) from a random curve point, even if you already know the values of \(A,B\) where \(A=aG, B=bG\).</p>
<p>The intended consequence of this reasoning (and notice how slippery this logic is!) is to say: DDH is <em>actually </em>hard, therefore ElGamal is computationally hiding. Or: since it <em>is</em> believed that DDH is hard, it follows that "we" (some undefined group of cryptographers) believe that ElGamal, as a commitment scheme, is computationally hiding.</p>
<h3>Brief observation: ElGamal is homomorphic</h3>
<p>Notice how the description of the homomorphic (with respect to addition) property of the Pedersen commitment cross applies here; even though we have two curve points here, not one, the same linearity exists:</p>
<p>\(C_{EG}(x_1, r_1) + C_{EG}(x_2, r_2) = \)</p>
<p>\((x_1G + r_1H, r_1G) + (x_2G + r_2H, r_2G)\)</p>
<p>\( = ((x_1 + x_2)G + (r_1+r_2)H, (r_1+r_2)G)\)</p>
<p>\( = C_{EG}(x_1+x_2, r_1+r_2)\)</p>
<h2>An unpalatable tradeoff?</h2>
<p>So all of the above is the "behind the scenes" of the discussion you'll often see in public about <a href="https://elementsproject.org/features/confidentialtransactions/investigation">Confidential Transactions</a> in Bitcoin, specifically (not that the tradeoff doesn't apply in other systems like Monero of course).</p>
<p>We naturally choose the Pedersen commitment for Confidential Transactions, because it's more compact (remember  size of output space!). It's only one curve point as output. Confidential Transactions take up a nontrivial amount of extra space in a transaction, so it's natural to prefer Pedersen to ElGamal for that reason, even though, importantly, <span style="textdecoration: underline;">both have the necessary homomorphic property</span> as already outlined.</p>
<p>Moreover (and more importantly, actually), a CT output needs a <em>range proof</em> (as explained in great detail e.g. <a href="https://github.com/AdamISZ/ConfidentialTransactionsDoc/">here</a>, see also bulletproofs e.g. <a href="https://eprint.iacr.org/2017/1066.pdf">here</a> and <a href="https://github.com/AdamISZ/from0k2bp">here</a>), which itself requires a <em>lot</em> of space  the range proofs described in the link, especially bulletproofs, go to a lot of trouble to condense this data to the greatest extent possible, since it must be published on the blockchain for all to verify, but that space usage is a serious issue.</p>
<p>The previous links point to all the work done on space optimisation for Pedersen; if we switched to ElGamal we'd lose that (I'm not exactly sure <em>where</em> we'd be in terms of how much space a CT style output would take up, but it would definitely be considerably more. While writing this I've noticed Andreev has written up an ElGamal range proof <a href="https://blog.chain.com/preparingforaquantumfuture45535b316314">here</a>).</p>
<p>Hence the title of the subsection; our choice in CT for something like Bitcoin seems to be:</p>
<ul>
<li>Continue on the existing path  highly space optimised Pedersen commitments with perfect hiding and computational binding under the ECDLP assumption.</li>
<li>Switch to ElGamal commitments, with much more bloaty range proofs and commitments, which however have perfect binding and computational hiding (under DDH assumption).</li>
</ul>
<p>Some people might argue that there is just too much fuss and worry about this. Computational is good enough, if our crypto hardness assumptions are good enough, and they are kind of industry standard already. However there's a big problem with this reasoning, and it was explained in the "tradeoffs" section of <a href="https://joinmarket.me/blog/blog/thesteganographicprinciple">this</a> earlier blog post. To avoid getting sidetracked on that now, let me summarize simply:</p>
<blockquote>
<p><em>A break in the binding assumption of Confidential Transactions can result in the attacker being able to print money in arbitrary amounts at any time, with absolutely no knowledge by the outside world.</em></p>
</blockquote>
<p>As I was at pains to point out in the linked blog post, this problem is not CTspecific; it's generic to any blinding mechanism relying on cryptographic hardness assumptions (i.e. without <strong>perfect binding</strong> or something analogous where even an infinitely powerful adversary cannot violate the binding of the blinded amount).</p>
<p>But here (for the rest of this blog post) we'll focus specifically on the CT version of the problem.</p>
<h2>The unexcluded middle</h2>
<p>If perfect binding and hiding are logically incompatible in a commitment, our only choice to violate the principle of the excluded middle is to step outside the boundaries of the problem described, and the most natural way to do that is to use two different commitments.</p>
<p>Using both Pedersen and ElGamal concurrently makes so little sense as to be incoherent, not least because an ElGamal commitment <em>contains</em> a Pedersen commitment. But the key word you could have skipped over in that sentence was <strong>concurrently</strong>. Ruffing and Malavolta in <a href="https://eprint.iacr.org/2017/237.pdf">this paper</a> suggest spreading the problem over time:</p>
<h2>Switch commitments</h2>
<p>The idea here is deceptively simple: what if you use an ElGamal commitment, but don't verify the nonPedersen component (the second point \(rG\) to use consistent notation) initially. If there is some time \(T\) at which all participants in the system agree that the ECDLP has "fallen" to quantum computing (the most well discussed failure vector of elliptic curve crypto), it could be required that after that flag day, spending of coins (presumably into some safer new cryptosystem defined by consensus; spending into currentstyle Bitcoin outputs would probably not make sense, here) is only valid if the verification/opening (and the range proof) were applied to the full ElGamal commitment \(xG+rH, rG\) and not just \(xG+rH\) as was allowed before \(T\).</p>
<p>There are two critiques that may immediately spring to mind, one obvious and one not:</p>
<ul>
<li>Not necessarily a realistic scenario  the break may be very public or not, it may be very gradual or not. Declaring a flag day is mostly assuming it being public. So it's not a panacea.</li>
<li>If you've been reading closely all this time, you'll be alert to a serious drawback: publishing an ElGamal commitment will not actually be hiding, if ECDLP is "cracked" (you remember that it requires DDH hardness, but it's easy to see that if you "crack" ECDLP you also crack DDH).</li>
</ul>
<p>Taking a more positive perspective, though: it's not as if \(T\) has to be a "panic stations day". Just as hash functions and TLS versions are sometimes retired because they <em>start</em> to show just a <em>tiny</em> bit of weakness, it would similarly make perfect sense for Bitcoin to be similarly prompt in making a switch to a postquantum cryptosystem once EC came into question, and not wait to be attacked. Not to say it would be easy!</p>
<p>This approach is sometimes called "cryptographic agility"  awkward as it seems, we do kinda want the ability to upgrade cryptographic protocols "inflight", while they are being used.</p>
<p>So at this point we have an ingenious and smart <em>amelioration</em> to the problem, but it can't be called a complete solution, I think  and principally because of the (admittedly tiny) possibility of a private break by some lone genius or similar.</p>
<h3>We put a commitment inside your commitment, so ...</h3>
<p>The authors and collaborators of the switch commitment paper and idea (Ruffing, Malavolta, Wuille, Poelstra, others .. I'm not actually sure) found a way to slightly improve the properties of such switch commitments: a structure they call the <strong>optin switch commitment</strong> which looks something like this:</p>
<p>\(xG + (r+\mathbb{H}(xG+rH  rG))H = xG + r'H\)</p>
<p>The idea is to tweak the blinding component of a standard Pedersen commitment with the hash of an ElGamal commitment to the same value (insert old meme as appropriate). Those of you aware of such things may instantly recognize a close parallel with ideas like paytocontract and <a href="https://bitcoinmagazine.com/articles/taprootcomingwhatitandhowitwillbenefitbitcoin/">taproot</a> (the latter was inspired by the former, so no surprise there). We're effectively committing to a "contract" which here is a promise to open to an ElGamal commitment <em>later,</em> if the consensus calls for it, while for now not revealing that contract, as it's hidden/blinded with the value \(r\).</p>
<p>As noted <a href="https://lists.launchpad.net/mimblewimble/msg00479.html">on the mimblewimble mailing list</a> by Ruffing, this has a couple of very important advantages over the nonoptin version:</p>
<ul>
<li>It preserves the perfect hiding of the Pedersen commitment for as long as the flag day \(T\) isn't reached (it's exactly a Pedersen commitment until then).</li>
<li>It doesn't use up another curve point on the blockchain  you only publish the single curve point as per Pedersen, and not two as per ElGamal.</li>
</ul>
<p>(Another useful feature  you can derive the value of \(r\) from your private key deterministically to make it more practical).</p>
<p>Of course one must prove it's secure (under the random oracle model) but for now I'll take that as a given (it's too much detail for here). But clearly this is a neat way to encapsulate that "switch" idea; modulo security proofs, it's an unqualified and very substantial improvement over the "naked ElGamal" version.</p>
<h3>A hard decision for the sleepy or lazy</h3>
<p>There is still an area of imperfection even in this soupedup "optin" switch commitment case. After the flag day \(T\) if you still have not moved coins from existing outputs, you can publish the ElGamal "contract" (commitment) inside the hash, thus keeping the binding property, so that the envisioned attackerpossessingaquantumcomputer will still not be able to print money, but in so doing, you give up the hiding (the value is revealed <em>at least to such attackers</em> because they can break the DDH problem). So thus a person failing to take action before said deadline \(T\) has at least to risk, and probably lose, one of those two: their privacy of amount or their money.</p>
<h2>Have your cake and eat it?</h2>
<p>Is it possible to do better than such a transition approach, as envisaged in the switch commitments paradigm?</p>
<p>As was earlier discussed, it suffers from not covering every threat scenario, in particular, it does not cover the scenario of a private and unexpected break.</p>
<p>Unfortunately this is where this very long blog post trails off ... because I don't know, and currently I don't think anyone else does.</p>
<p>My personal feeling was that the switch commitment paradigm suggests there might be a way to finesse this tradeoff about using commitments. And it also seems to be something which Adam Back seems to have gone some way to thinking through  the fact that a single commitment scheme can't provide perfect hiding and binding for a single value doesn't imply that it is impossible to get this property, <strong>as long you're not working with the same value</strong>. For example, what if you could provide an ElGamal commitment for the money created in a Bitcoin <em>block</em>, while providing Pedersen commitments as in the current design of CT for the individual transactions? This means that a quantum or ECDLP breaking attacker can "snoop" into the overall value created in a block, but this should either be already known or uninteresting, while although he could in theory violate the binding property of individual transactions, this would in turn violate the binding of the blocklevel commitment which is supposed to be impossible?</p>
<p>I suspect my original line of thinking is somehow incoherent (how, mathematically, are the blocklevel and transactionlevel commitments related?), but Dr Back seems to have in mind something involving coinjoinlike interactivity. I am leaving it here without attempting to describe further, because the question seems to continue to be interesting and if there <em>is</em> a solution (even perhaps if it involves interactivity), it would be a hugely important fact, making CT a much more plausible technology for a global money.</p>
<h3>Build the wall?</h3>
<p>We could also take the Trumpian approach  it's far from infeasible to imagine that there is a mechanism that prevents CT coins arriving back into plaintext area without allowing any hidden inflation that <em>might</em> occur to "infect". This is essentially the sidechain model, except it could be implemented in a variety of different ways. In fact, this <strong>model already does exist</strong> in the sidechain <a href="https://blockstream.com/liquid/">Liquid</a>, which uses CT. But there have also been proposals to implement CT as a kind of extension block (which has slightly different tradeoffs to a sidechain), for example see ZmnSCPxj's note <a href="https://lists.linuxfoundation.org/pipermail/bitcoindev/2019January/016605.html">here</a> .</p>
<p></p>
<p></p>PayJoin20190117T16:48:55+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/payjoin/<h2>PayJoin.</h2>
<p></p>
<p>You haven't read any other blog posts here? No worries, here's what you need to know (<em>unless you're an expert, read them anyway...</em>):</p>
<ul>
<li>A utxo is an "unspent transaction output"  a Bitcoin transaction creates one or more of these, and each contains a specific amount of Bitcoin. Those outputs get "used up" in the transaction that spends them (somewhat like physical coins, someone gives you them in a payment to you, then you give them to someone else when you spend them; bitcoins aren't coins, utxos are coins; only difference is physical coins don't get destroyed in transactions).</li>
<li>The fees you have to pay for a Bitcoin transaction depend on how many bytes it takes up; this is *somewhat* dominated by how many inputs you provide, although there are other factors.</li>
<li>CoinJoin basically means  two or more people provide inputs (utxos) to a transaction and cosign without needing trust, because when they sign that the output amounts and addresses are what they expect. <strong>Note that CoinJoin requires interaction, almost always.</strong></li>
<li>Traditional "equalsized" CoinJoin means a bunch of people paying <em>themselves</em> the same fixed amount in a single transaction (according to the process just mentioned), with the intention that nobody can tell which of the equal sized outputs belong to who (basically!).</li>
</ul>
<h2>The drawbacks of CoinJoin as implemented</h2>
<p>Current implementations of CoinJoin are of the "equalsized" variety (see above). This requires coordination, but it's possible to get a decent number of people to come together and agree to do a CoinJoin of a certain fixed amount. The negative is that this kind of transaction is trivially distinguishable from an "ordinary" transaction, in particular a payment from one counterparty to another. Here's a typical Joinmarket CoinJoin (and other implementations are just as, or more, distinguishable):</p>
<p></p>
<p><img alt="Equaloutscoinjoinexample" height="433" src="https://joinmarket.me/static/media/uploads/.thumbnails/screenshot_from_20190118_150033.png/screenshot_from_20190118_150033807x433.png" width="807"/></p>
<p>The biggest flag of "this is CoinJoin" is exactly the multiple equalvalue (0.18875417 here) outputs that are the core premise of the idea, that give the anonymity. Here, you get anonymity in an "anonymity set" of all participants of <em>this</em> transaction, first, but through repeated rounds, you <em>kind of</em> get a much bigger anonymity set, ultimately of all participants of that CoinJoin implementation in the absolute best scenario. But it's still only a small chunk of Bitcoin usage generally.</p>
<p>And while this obviously gets better if more people use it, there is a limit to that thinking: because <strong>all participants are forced to use the same denomination for any single round</strong>, it isn't possible to fold in the payments you're doing using Bitcoin as a currency (don't laugh!) into these CoinJoin rounds (notice: this problem mostly disappears with blinded amounts).</p>
<p>So a world where "basically everyone uses CoinJoin" is cool for privacy, but could end up pretty bad for scalability, because these transactions are <em>in addition to</em> the normal payments.</p>
<p>Also, the fact that these transactions are trivially watermarked means that, if the blockchain analyst is not able to "crack" and unmix such transactions, he can at least isolate them in analysis. That's something; "these coins went from Exchange A to wallet B and then into this mixset" may be a somewhat negative result, but it's still a result. There are even noises made occasionally that coins might be blocked from being sent to certain exchangetype entities if they're seen to have come from a "mixer" (doesn't matter that CoinJoin is <em>trustless</em> mixing here; just that it's an activity specific for obfuscation).</p>
<p>I don't mean to scaremonger  I have used such CoinJoin for years (measured in the thousands) and will continue to do so, and never had payments blocked because of it. But this is another angle that must be borne in mind.</p>
<p>So let's say our primary goal is to minimize the negative privacy effects of blockchain analysis; can we do better? It's debatable, but we <em>do</em> have another angle of attack.</p>
<p></p>
<h2>Hiding in a much bigger crowd ... ?</h2>
<p><span style="textdecoration: underline;">One angle is to make your behaviour look more like other, noncoinjoin transactions</span>. (For the philosophically/abstract inclined people, <a href="https://joinmarket.me/blog/blog/thesteganographicprinciple/">this post might be of interest</a>, but it sidetracks us here, so  later!). Let's think of the naive way to do that. Suppose just Alice and Bob make a 2 party CoinJoin:</p>
<p><code>0.05 BTC > 0.05 BTC 3AliceSAddReSs</code></p>
<p><code>0.05 BTC > 0.05 BTC 3BobSAddReSs</code></p>
<p>This first attempt is a clear failure  it "looks like an ordinary payment" <em>only</em> in the sense that it has two outputs (one change, one payment). But the failure is not <em>just</em> the obvious, that the output amounts are equal and so "obviously CoinJoin". There's another aspect of that failure, illustrated here:</p>
<p><code>0.01 BTC > 0.05 BTC 3AliceSAddReSs</code></p>
<p><code>0.04 BTC > 0.06 BTC 3BobSAddReSs</code></p>
<p><code>0.03 BTC > </code></p>
<p><code>0.03 BTC ></code></p>
<p>This at least is <em>more</em> plausible as a payment, but it shows the <strong>subset sum</strong> problem that I was describing in my <a href="https://joinmarket.me/blog/blog/coinjoinxt/">CoinJoinXT post</a>  and trying to solve with CoinJoinUnlimited (i.e. using a Lightning channel to break the subset sum problem and feedback the LN privacy onto the main chain). While the blockchain analyst <em>could </em>interpret this as a payment, semireasonably, of 0.05 btc by one participant, he could also notice that there are two subsets of the inputs that add up to 0.05, 0.06. And also splitting the outputs doesn't fundamentally solve that problem, notice (they'd also have to split into subsets), and it would anyway break the idea of "looking like a normal payment" (one payment, one change):</p>
<p><code>0.01 BTC > 0.011 BTC 3AliceSAddReSs</code></p>
<p><code>0.04 BTC > 0.022 BTC 3BobSAddReSs</code></p>
<p><code>0.03 BTC > 0.039 BTC 3Alice2</code></p>
<p><code>0.03 BTC > 0.038 BTC 3Bob2</code></p>
<p></p>
<p>After you think about this problem for a while you come to the conclusion  only if there's actually a transfer of coins from one party to the other is it solved. Hence <a href="https://joinmarket.me/blog/blog/coinjoinxt/">CoinJoinXT</a>.</p>
<p>But also, hence <strong>PayJoin</strong>  why not actually do a CoinJoin <span style="textdecoration: underline;">while you are making a payment?</span></p>
<p><span style="textdecoration: underline;"></span></p>
<h2>PayJoin advantages</h2>
<p>I'm not sure who first thought of doing CoinJoins (see bullet point at start) of this particular flavour, but a <a href="https://blockstream.com/2018/08/08/improvingprivacyusingpaytoendpoint/">blogpost from Matthew Haywood</a> last summer detailed an implementation approach which came out of a technical workshop in London shortly before, and a little later a <a href="https://github.com/bitcoin/bips/blob/master/bip0079.mediawiki">BIP</a> was put out by Ryan Havar.</p>
<p></p>
<p>The central idea is:</p>
<ul>
<li>Let Bob do a CoinJoin with his customer Alice  he'll provide at least one utxo as input, and that/those utxos will be consumed, meaning that in net, he will have no more utxos after the transaction than before, and an obfuscation of ownership of the inputs will have happened <span style="textdecoration: underline;">without it looking different from an ordinary payment.</span></li>
</ul>
<p></p>
<p>Before we look in detail at the advantages, it's worth answering my earlier question ("Why not actually do a CoinJoin while you are making a payment?") in the negative: it's not easy to coordinate that. It means that either (a) all wallets support it and have a way for *anyone* to connect to *anyone* to negotiate this (2party) CoinJoin or (b) it's only limited to peer to peer payments between owners of a specific wallet that has a method for them to communicate. So let's be clear: this is not going to suddently take over the world, but incremental increases in usage could be tremendously valuable (I'll explain that statement shortly; but you probably already get it).<span style="textdecoration: underline;"></span></p>
<ul>
<li><strong>Advantage 1: Hiding the payment amount</strong></li>
</ul>
<p>This is what will immediately stand out from looking at the idea. Bob "chips in" a utxo (or sometimes more than one). So the payment <em>output</em> will be more than the actual payment, and it will be profoundly unobvious what the true payment amount was. Here's an example:</p>
<p><code>0.05 BTC > 0.04 BTC 3AliceSAddReSs</code></p>
<p><code>0.09 BTC > 0.18 BTC 3BobSAddReSs</code></p>
<p><code>0.08 BTC ></code></p>
<p>Now, actually, Alice paid Bob 0.1 BTC using 0.09 and 0.05, getting back 0.04 change. But what does a blockchain analyst think? His first interpretation will certainly be that there is a payment <em>either</em> of 0.04 BTC or 0.18 BTC, by the owner of the wallet containing all the inputs. Now, it probably seems very unlikely that the <em>payment</em> was 0.04 and the <em>change </em>0.18. Why? Because, if the payment output were 0.04, why would you use all three of those utxos, and not just the first, say? (0.05). This line of reasoning we have called "UIH1" in the comments to <a href="https://gist.github.com/AdamISZ/4551b947789d3216bacfcb7af25e029e">this gist</a> (h/t Chris Belcher for the nomenclature  "unnecessary input heuristic") for the details. To be fair, this kind of deduction by a blockchain analyst is unreliable, as it depends on wallet selection algorithms; many are not nearly so simplistic that this deduction would be correct. But possibly combined with wallet fingerprinting and detailed knowledge of wallet selection algorithms, it's one very reasonable line of attack to finding the change output and hence the payment output.</p>
<p>For those interested in the "weeds" I've reproduced the key points about this UIH1 and UIH2 (probably more important) including stats collected by LaurentMT of oxt.me, in an "Appendix" section at the end of this post.</p>
<p>Anyway, what else <em>could </em>the payment amount be, in the transaction above? As well as 0.04 and 0.18, there is 0.09 and 0.01. Do you see the reasoning? <em>If </em>we assume that PayJoin is a possibility, then one party could be consuming 0.09 and 0.08 and getting back 0.01. And similarly for other contributions of inputs. In the simplest case, I would claim there are 4 potential payment amounts if there are only two inputs and we assume that one of the two is owned by the receiver. For the blockchain analyst, this is a huge mess.</p>
<ul>
<li><strong>Advantage 2  breaking Heuristic 1</strong></li>
</ul>
<p>I discussed Heuristic 1 in the <a href="https://joinmarket.me/blog/blog/feeds/atom/%22https://joinmarket.me/blog/blog/coinjoinxt/">CoinJoinXT post</a>. Simple description: people (analysts) assume that all the inputs to any particular transaction are owned by one wallet/owner; i.e. they assume coinjoin is not used, usually. Following the overall logic of our narrative here, it's obvious what the main point is with PayJoin  we break the heuristic <em>without flagging to the external observer that the breakage has occurred. <strong></strong></em>This is enormously important, even if the breakage of the assumption of common input ownership on its own seems rather trivial (especially if PayJoin is used by only few people), with only 2 counterparties in each transaction.</p>
<ul>
<li><strong>Advantage 3  Utxo sanitization</strong></li>
</ul>
<p>This one might not occur to you immediately, at all, but is actually really nice. Consider the plight of the merchant who sells 1,000 widgest per day for Bitcoin. At the end of the day he has 1,000 utxos that he has to spend. Perhaps the next day he pays his supplier with 80% of the money; he'll have to construct a transaction (crudest scenario) with 800 inputs. It's not just that that costs a lot in fees (it does!); we can't really directly solve that problem (well  use layer 2!  but that's another blog post); but we can solve something else about it  the privacy. The merchant immediately links <em>almost </em><em>all</em> of his payments in the 800input payout transaction  horrible!</p>
<p>But PayJoin really helps this; each payment that comes in can consume the utxo of the last payment. Here are two fictitious widget payments in sequence to illustrate; Bob's utxos are bolded for clarity:</p>
<p><span style="textdecoration: underline;">PayJoin 1  Alice pays Bob 0.1 for a widget:</span></p>
<p><code>0.05 BTC > 0.04 BTC 3AliceSAddReSs</code></p>
<p><code>0.09 BTC > <strong>0.18 BTC</strong> 3BobSAddReSs</code></p>
<p><code><strong>0.08 BTC</strong> ></code></p>
<p>(notice: Bob used up one utxo and created one utxo  no net change)</p>
<p><span style="textdecoration: underline;">PayJoin2  Carol pays Bob 0.05 for a discount widget:</span></p>
<p><code>0.01 BTC > 0.02 BTC 3CarolSAddReSs</code></p>
<p><code>0.06 BTC > <strong>0.23 BTC</strong> 3BobSAddReSs</code></p>
<p><code><strong>0.18 BTC</strong> ></code></p>
<p>This would be a kind of snowball utxo in the naive interpretation, that gets bigger and bigger with each payment. In the fantasy case of every payment being PayJoin, the merchant has a particularly easy wallet to deal with  a wallet that only ever has 1 coin/utxo! (I know it's quite dubious to think that nobody could trace this sequence, there are other potential giveaways <em>in this case</em> than just Heuristic 1; but with Heuristic 1 gone, you have a lot more room to breathe, privacywise).</p>
<p>It's worth mentioning though that the full snowball effect can damage the anonymity set: after several such transactions, Bob's utxo is starting to get big, and may dwarf other utxos used in the transaction. In this case, the transaction will violate "UIH2" (you may remember UIH1  again, see the Appendix for more details on this) because a wallet <em>probably </em>wouldn't choose other utxos if it can fulfil the payment with only one. So this may create a dynamic where it's better to mix PayJoin with nonPayJoin payments.</p>
<ul>
<li><strong>Advantage 4  hiding in (and being helpful to) the large crowd</strong></li>
</ul>
<p>"...but incremental increases in usage could be tremendously valuable..."  let's be explicit about that now. If you're even reasonably careful, these PayJoin transactions will be basically indistinguishable from ordinary payments (see earlier comments about UIH1 and UIH2 here, which don't contradict this statement). It's a good idea to use decide on a specific locktime and sequence value that fits in with commonly used wallets (transaction version 2 makes the most sense). Now, here's the cool thing: suppose a smallish uptake of this was publically observed. Let's say 5% of payments used this method. <strong>The point is that nobody will know which 5% of payments are PayJoin</strong>. That is a great achievement (one that we're not yet ready to achieve for some other privacy techniques which use custom scripts, for example; that may happen after Schnorr/taproot but not yet), because <em>it means that all payments, including ones that don't use PayJoin, gain a privacy advantage!</em></p>
<h2>Merchants? Automation?</h2>
<p>The aforementioned <a href="https://github.com/bitcoin/bips/blob/master/bip0079.mediawiki">BIP79</a> tries to address how this might work in a standardized protocol; there's probably still significant work to do before the becomes actualized. As it stands, it may be enough to have the following features:</p>
<ul>
<li>Some kind of "endpoint" (hence "pay to endpoint"/p2ep) that a customer/payer can connect to encoded as some kind of URL. A Tor hidden service would be ideal, in some cases. It could be encoded in the payment request similar to BIP21 for example.</li>
<li>Some safety measures on the server side (the merchant/receiver) to make sure that an attacker doesn't use the service to connect, request, and block: thus enumerating the server's (merchant's) utxos. BIP79 has given one defensive measure against this that may be sufficient, Haywood's blog post discussed some more advanced ideas on that score.</li>
<li>To state the obvious friction point  wallets would have to implement such a thing, and it is not trivial compared to features like RBF which are pure Bitcoin.</li>
</ul>
<p></p>
<h2>Who pays the fees?</h2>
<p>The "snowball effect" described above, where the merchant always has one utxo, may lead you to think that we are saving a lot of fees (no 800 input transactions). But not true except because of some second/third order effect: every payment to the merchant creates a utxo, and every one of those must be paid for in fees when consumed in some transaction. The effect here is to pay those fees slowly over time. And it's left open to the implementation how to distribute the bitcoin transaction fees of the CoinJoin. Most logically, each participant pays according to the amount of utxos they consume; I leave the question open here.</p>
<h2>Implementation in practice</h2>
<p>As far as I know as of this writing (midJanuary 2019), there are two implementations of this idea in the wild. One is from Samourai Wallet, called <a href="https://samouraiwallet.com/stowaway">Stowaway</a> and the other is in <a href="https://github.com/JoinmarketOrg/joinmarketclientserver/blob/master/docs/PAYJOIN.md">Joinmarket</a> as of version 0.5.2 (just released).</p>
<p>I gave a demo of the latter in my last <a href="https://joinmarket.me/blog/blog/payjoinbasicdemo/">post on this blog</a>.</p>
<p>In both cases this is intended for peers to pay each other, i.e. it's not something for large scale merchant automation (as per discussion in previous section).</p>
<p>It requires communication between parties, as does any CoinJoin, except arguably <a href="https://joinmarket.me/blog/blog/snicker/">SNICKER</a>.</p>
<p>The sender of the payment always sends a nonCoinJoin payment transaction to start with; it's a convenient/sane thing to do, because if connection problems occur, or software problems, the receiver can simply broadcast this "fallback" payment instead.</p>
<p>In Joinmarket specifically, the implementation looks crudely like this:</p>
<p><code>Sender Receiver</code></p>
<p><code>pubkey+versionrange ></code></p>
<p><code> < pubkey and version</code></p>
<p><code>(ECDH e2e encryption set up)</code></p>
<p><code>fallback tx ></code></p>
<p><code> < PayJoin tx partialsigned</code></p>
<p><code>cosigns and broadcasts</code></p>
<p>Before starting that interchange of course, the receiver must "send" (somehow) the sender the payment amount and destination address, as well as (in Joinmarket) an ephemeral "nick" to communicate over the message channel. Details here of course will vary, but bear in mind that as any normal payment, there <em>must </em>be some mechanism for receiver to communicate payment information to the sender.</p>
<p></p>
<h2>Conclusion</h2>
<p>This is another nail in the coffin of blockchain analysis. If 5% of us do this, it will <em>not</em> be safe to assume that a totally ordinary looking payment is not a CoinJoin. That's basically it.</p>
<p></p>
<p></p>
<h3>Appendix: Unnecessary Input Heuristics</h3>
<p>The health warning to this reasoning has already been given: wallets will definitely not <em>always </em>respect the logic given below  I know of at least one such case (h/t David Harding). However I think it's worth paying attention to (this is slightly edited from the comment section of the referenced gist):</p>
<p> <span style="textdecoration: underline;">Definitions:</span></p>
<p>"UIH1" : one output is smaller than any input. This heuristically implies that <em>that</em> output is not a payment, and must therefore be a change output.</p>
<p>"UIH2": one input is larger than any output. This heuristically implies that <em>no output</em> is a payment, or, to say it better, it implies that this is not a normal walletcreated payment, it's something strange/exotic.</p>
<p>Note: UIH2 does not necessarily imply UIH1.</p>
<p><del> </del></p>
<p>So we just have to focus on UIH2. Avoiding UIH1 condition is nice, because it means that both outputs could be the payment; but in any case the normal blockchain analysis will be wrong about the payment amount. If we don't avoid the UIH2 condition, though, we lose the steganographic aspect which is at least 50% of the appeal of this technique.</p>
<p>Joinmarket's current implementation does its best to avoid UIH2, but proceeds with PayJoin anyway even if it can't. The reasoning is partially as already discussed: not all wallets follow this logic; the other part of the reasoning is the actual data, as we see next:</p>
<p><span style="textdecoration: underline;">Data collection from LaurentMT:</span></p>
<p>From block 552084 to block 552207 (One day: 01/12/2018)</p>
<ul>
<li>Txs with 2 outputs and more than 1 input = 35,349
<ul>
<li>UIH1 Txs (identifiable change output) = 19,020 (0.54)</li>
<li>!UIH1 Txs = 16,203 (0.46)</li>
<li>Ambiguous Txs = 126 (0.00)</li>
</ul>
</li>
</ul>
<p>From block 552322 to block 553207 (One week: 03/12/2018  09/12/2018)</p>
<ul>
<li>Txs with 2 outputs and more than 1 input = 268,092
<ul>
<li>UIH1 Txs (identifiable change output) = 145,264 (0.54)</li>
<li>!UIH1 Txs = 121,820 (0.45)</li>
<li>Ambiguous Txs = 1,008 (0.00)</li>
</ul>
</li>
</ul>
<p>And here are a few stats for UIH2:</p>
<p>Stats from block 552084 to block 552207 (One day: 01/12/2018)</p>
<ul>
<li>Txs with 2 outputs and more than 1 input = 35,349
<ul>
<li>UIH2 Txs = 10,986 (0.31)</li>
<li>!UIH2 Txs = 23,596 (0.67)</li>
<li>Ambiguous Txs = 767 (0.02)</li>
</ul>
</li>
</ul>
<p>From block 552322 to block 553207 (One week: 03/12/2018  09/12/2018)</p>
<ul>
<li>Txs with 2 outputs and more than 1 input = 268,092
<ul>
<li>UIH2 Txs = 83,513 (0.31)</li>
<li>!UIH2 Txs = 178,638 (0.67)</li>
<li>Ambiguous Txs = 5,941 (0.02)</li>
</ul>
</li>
</ul>
<p></p>Payjoin  a basic demo20190112T21:38:35+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/payjoinbasicdemo/<p>I'll shortly be writing a blog post about the idea of "PayJoin", which is a variant of what's sometimes been called "PayToEndpoint" coinjoin, or "p2ep".</p>
<p><strong>If you don't know what "p2ep" is, you're probably better off waiting for the next blogpost.</strong></p>
<p>For now I just wanted to show an example of how it runs in Joinmarket (<a href="https://github.com/JoinMarketOrg/joinmarketclientserver/pull/279">PR here</a> ; not yet merged but won't be long).</p>
<p></p>
<p>This is a terminal recording with asciinema (which is much more lightweight than a video).</p>
<p>Here, I ran both sender and receiver; each terminal's playback is shown sidebyside at the below link.</p>
<p>Sorry for the copypaste fumbles in the middle :)</p>
<p></p>
<p><a href="https://asciinema.org/a/221153?speed=2"><img height="409" src="https://asciinema.org/a/221153.png" width="727"/></a></p>
<p>The resulting transaction can be seen <a href="https://www.smartbit.com.au/tx/8cb0af96f1a2693683621758acbf3b7a7ad69a69672c61e144941d666f72da2a">here</a>.</p>
<p>As a fun exercise, you could challenge your friends by showing them this transaction and asking them:</p>
<p>"What is the payment amount?"</p>
<p>Having seen the above, you'll know, but I bet they won't be able to figure it out :)</p>
<p>If you do have such a discussion, you'll probably want to note that <span style="textdecoration: underline;">none of the inputs are bigger than the</span></p>
<p><span style="textdecoration: underline;">biggest output</span>, so there isn't really a reason to be suspicious (for an outsider) that this <em>is</em> anything other</p>
<p>than an ordinary payment; or is there? There's more to discuss for sure :)</p>
<p></p>
<p>Samourai are working on something very similar; <a href="https://mastodon.social/web/statuses/101406122076731312">this</a> discussion may be interesting too!</p>
<p><a href="https://gist.github.com/AdamISZ/4551b947789d3216bacfcb7af25e029e">Gist</a> discussing some technical issues (see the comments in particular  about UIH1 and UIH2).</p>
<p>Questions? You can reach me on IRC freenode #joinmarket or on <a href="https://mastodon.social/web/accounts/406755">mastodon</a>.</p>
<p>If anyone wants to give it a try in the next week or two, feel free to contact me there.</p>CoinJoinXT20180630T16:38:18+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/coinjoinxt/<h1>CoinJoinXT  a more flexible, extended approach to CoinJoin</h1>
<p><em>Ideas were first discussed <a href="https://gist.github.com/AdamISZ/a5b3fcdd8de4575dbb8e5fba8a9bd88c">here</a>. Thanks again to arubi on IRC for helping me flesh them out.<br/></em></p>
<h2>Introduction</h2>
<p>We assume that the reader is familiar with CoinJoin as a basic idea  collaboratively providing inputs to a transactions so that it may be made difficult or impossible to distinguish ownership/control of the outputs.</p>
<p>The way that CoinJoin is used in practice is (today mainly using JoinMarket, but others over Bitcoin's history) is to create largeish transactions with multiple outputs of exactly the same amount. This can be called an "intrinsic fungibility" model  since, although the transactions created are unambiguously recognizable as CoinJoins, the indistinguishability of said equal outputs is kind of "absolute".</p>
<p>However, as partially discussed in the earlier blog post <a href="https://joinmarket.me/blog/blog/thesteganographicprinciple/">"the steganographic principle"</a>, there's at least an argument for creating fungibility in a less explicit way  that is to say, creating transactions that have a fungibility effect but aren't <em>necessarily</em> visible as such  they <em>may</em> look like ordinary payments. I'll call this the <em>deniability</em> model vs the <em>intrinsic fungibility</em> model. It's harder to make this work, but it has the possibility of being much more effective than the <em>intrinsic fungibility model</em>, since it gives the adversary (who we'll talk about in a minute) an additional, huge problem: he doesn't even know where to start.</p>
<h2>The adversary's assumptions</h2>
<p>In trying to create privacy, we treat the "blockchain analyst" as our adversary (henceforth just "A").</p>
<p>Blockchain analysis consists, perhaps, of two broad areas (not sure there is any canonical definition); we can call the first one "metadata", vaguely, and think of it is every kind of data that is not directly recorded on the blockchain, such as personally identifying information, exchange records etc, network info etc. In practice, it's probably the most important. The second is stuff recorded directly on the blockchain  pseudonyms (scriptPubKeys/addresses) and amount information (on nonamountblinded blockchains as Bitcoin's is currently; for a discussion about that see this earlier <a href="https://joinmarket.me/blog/blog/thesteganographicprinciple/">blog post</a>); note that amount information includes the implicit amount  network fee.</p>
<p>Timing information perhaps straddles the two categories, because while transactions are (loosely) timestamped, there is also the business of trying to pick up timing and perhaps geographic information from snooping the P2P network.</p>
<p>With regard to that second category, the main goal of A is to correlate ownership of different utxos. An old <a href="https://cseweb.ucsd.edu/~smeiklejohn/files/imc13.pdf">paper</a> of Meiklejohn et al 2013 identified two Heuristics (let's call them probabilistic assumptions), of which the first was by far the most important:</p>
<ul>
<li>Heuristic 1  All inputs to a transaction are owned by the same party</li>
<li>Heuristic 2  Onetime change addresses are owned by the same party as the inputs</li>
</ul>
<p>The second is less important mainly because it had to be caveated quite a bit and wasn't reliable in naive form; but, identification of change addresses generally is a plausible angle for A. The first has been, as far as I know, the bedrock of blockchain analysis and has been referred to in many other papers, was mentioned in Satoshi's whitepaper, and you can see one functional example at the longexistent website <a href="https://www.walletexplorer.com">walletexplorer</a>.</p>
<p><span style="textdecoration: underline;">But I think it's important to observe that this list is incomplete.</span><span style="textdecoration: underline;"></span></p>
<p>I'll now add two more items to the list; the first is omitted because it's elementary, the other, because it's subtle (and indeed you might find it a bit dumb at first sight):</p>
<ul>
<li><code>Heuristic/Assumption 0</code>: All inputs controlled by only one pubkey are unilaterally controlled</li>
<li>Heuristic/Assumption 1: All inputs to a transaction are owned by the same party</li>
<li>Heuristic/Assumption 2(?): Onetime change addresses are owned by the same party as the inputs</li>
<li><code>Heuristic/Assumption 3</code>: Transfer of ownership between parties in one transaction implies payment</li>
</ul>
<p>So, "Heuristic/Assumption" because assumption is probably a better word for all of these generally, but I want to keep the existing nomenclature, the "?" for 2 is simply because, as mentioned, this one is problematic (although still worthy of consideration).</p>
<p><strong>Assumption 0</strong>: basically, that if it's not multisig, was never fully safe; there was always <a href="https://en.wikipedia.org/wiki/Shamir's_Secret_Sharing">Shamir's secret sharing</a> to share shards of a key, albeit that's very rarely used, and you can argue pedantically that full reconstruction means unilateral control. But Assumption 0 is a lot less safe now due to the recent <a href="https://eprint.iacr.org/2018/472">work</a> by MorenoSanchez et al. which means, at the very least, that 2 parties can easily use a 2party computation based on the Paillier encryption system to effectively use a single ECDSA pubkey as a 22 multisig. So this assumption is generally unspoken, but in my opinion is now generally important (i.e. not necessarily correct!).</p>
<p><strong>Assumption 3</strong>: this is rather strange and looks tautological; I could have even written "transfer of ownership between parties in one transaction implies transfer of ownership" to be cheeky. The point, if it is not clear to you, will become clear when I explain what "CoinJoinXT" means.</p>
<p></p>
<p>Our purpose, now, is to make A's job harder <strong>by trying to invalidate all of the above assumptions at once</strong>.</p>
<h2>Quick refresher: BIP141</h2>
<p>This has been discussed in other blog posts about various types of "CoinSwap", so I won't dwell on it.</p>
<p>Segwit fixes transaction malleability (<a href="https://github.com/bitcoin/bips/blob/master/bip0141.mediawiki">BIP141</a>, along with BIP143,144 were the BIPs that specified segwit). One of the most important implications of this is explained directly in BIP 141 itself, to <a href="https://github.com/bitcoin/bips/blob/master/bip0141.mediawiki#Trustfree_unconfirmed_transaction_dependency_chain">quote</a> from it:</p>
<blockquote>
<p><em>Two parties, Alice and Bob, may agree to send certain amount of Bitcoin to a 2of2 multisig output (the "funding transaction"). Without signing the funding transaction, they may create another transaction, timelocked in the future, spending the 2of2 multisig output to third account(s) (the "spending transaction"). Alice and Bob will sign the spending transaction and exchange the signatures. After examining the signatures, they will sign and commit the funding transaction to the blockchain. Without further action, the spending transaction will be confirmed after the locktime and release the funding according to the original contract.</em></p>
</blockquote>
<p>In short, if we agree a transaction, then we can fix its txid and sign transactions which use its output(s). The BIP specifically references the Lightning Network as an example of the application of this pattern, but of course it's not restricted to it. We can have Alice and Bob agree to any arbitrary set of transactions and presign them, in advance, with all of them having the funding transaction as the root.</p>
<h2>CoinJoinXT  the basic case</h2>
<p>CoinJoin involves 2 or more parties contributing their utxos into 1 transaction, but using the above model they can do the same to a funding transaction, but then presign a set of more than one spending transaction. Here's a simple schematic:</p>
<pre><code>A 1btc >
F (2,2,A,B) +
B 1btc > 

+>[Proposed transaction graph (PTG) e.g. >TX1>TX2>TX3 ..]</code></pre>
<p>In human terms, you can envisage that: Alice and Bob would like to start to negotiate a set of conditional contracts about what happens to their money. Then they go through these steps:</p>
<ol>
<li>One side proposes F (the funding transaction) and a full graph of unsigned transactions to fill out the PTG above; e.g. Alice proposes, Bob and Alice share data (pubkeys, destination addresses). Note that the set doesn't have to be a chain (TX1>TX2>TX3...), it can be a tree, but each transaction must require signoff of both parties (either, at least one 22 multisig utxo, or at least one utxo whose key is owned by each party).</li>
<li>They exchange signatures on all transactions in the PTG, in either order. Of course, they abort if signatures don't validate.</li>
<li>With this in place (i.e. <strong>only</strong> after valid completion of (2)), they both sign (in either order) F.</li>
<li>Now both sides have a valid transaction set, starting with F. Either or both can broadcast them. <span style="textdecoration: underline;">The transactions are <em>all</em> guaranteed to occur as long as at least one of them wants it</span>. Contrariwise, <strong>none</strong> of them is valid without F being broadcast.</li>
</ol>
<p>This does achieve one significant thing: <strong>one transaction such as TX2 can transfer coins to, say, Bob's wallet, giving Alice nothing; and yet we can still get the overall effect of a CoinJoin. In other words, we've opened up the possibility to violate Heuristic 3 as well as Heuristic 1, in the same (short) interaction.</strong></p>
<p>This construction works fine if <em>all</em> inputs used in transactions in the PTG are descendants of F; but this makes the construction very limited. So we'll immediately add more details to allow a more general usecase, in the next section.</p>
<h2>Introducing Promises</h2>
<p>If we allowed any of the transactions (TX1, TX2, ...) in the PTG in our previous example to have an input which did <em>not</em> come from the funding transaction F, then we would have introduced a risk; if Alice added utxo UA to, say, TX2, then, before Bob attempted to broadcast TX2, she could double spend it. This would break the atomicity of the graph, which was what allowed the crucial additional interesting feature (in bold, above): that an individual transaction could transfer funds to one party, without risks to the other. To address this problem, we call these additional inputs <strong>promise utxos</strong> and make use of <strong>refund transactions</strong>.</p>
<pre><code>A 1btc >
F (2,2,A,B) 
B 1btc >  +> external payout 0.5 btc to Bob
 
+>[TX1 > TX2 > TX3 > TX4]
 ^
 
 
 + utxo A1

+> refund locktime M, pay out *remaining* funds to A: 1btc, B: 0.5btc</code></pre>
<p>In words: if, between the negotiation time and the time of broadcast of TX3, Alice spends A1 in some other transaction, Bob will still be safe; after block M he can simply broadcast the presigned refund transaction to claim the exact number of coins he is owed at that point in the graph.</p>
<p>The above addresses the case of a single external input being included in a chain of transactions in the PTG (here, TX1,2,3,4). Extending this, and generalising to allowing external inputs in many transactions, is straightforward; we can add such inPTG backouts at every step, redeeming all remaining funds to parties according to what they're owed.</p>
<p>To summarize this section and how it differs from the original, simpler construction:</p>
<p>Alice and Bob have a choice:</p>
<ol>
<li>They can set up a fully trustless PTG, without promises. They are then guaranteed to achieve "all or nothing": either all cooperative signing works, then all transactions can be broadcast (as long as <em>at least one</em> of them wants to), or nothing (including F) is broadcast at all.</li>
<li>They can set up a PTG including promises from one or both parties. Now they don't get "all or nothing" but only ensure that the transactions that complete are a subset, in order, from the start F. To achieve this they add presigned backouts at (probably every) step, so that if the chain "breaks" somewhere along, they will recover all the funds remaining that are owed to them.</li>
</ol>
<p>The tradeoff is: (2) is not perfectly atomic, but it allows the transaction graph to include utxos from outside of F's ancestory, particularly useful for privacy applications. In a sequence of 10 coinjoins, you may be happy to risk that TXs 610 don't end up happening, if it doesn't cost you money. Case (2) is more likely to be of interest.</p>
<h2>Interlude  overview of features of CoinJoinXT</h2>
<p>There's a large design space here.</p>
<ul>
<li>We can have N parties, not just 2.</li>
<li>We can have as many transactions as we like.</li>
<li>We can have a tree with F as root, rather than a chain.</li>
<li>We can have as many promise utxos from any of the N parties as we like.</li>
</ul>
<p>A mixture of these features may give different tradeoffs in terms of <em>intrinsic fungibility</em> vs <em>deniability </em>vs <em>cost</em>; the tradeoff discussed in the introduction.</p>
<p><strong>Interactivity</strong>  unlike either a CoinSwap of types discussed earlier in this blog, or doing multiple CoinJoins (to get a better fungibility effect than just a single one), this only requires one "phase" of interactivity (in terms of rounds, it may be 3). The two parties connect, exchange data and signatures, and then immediately disconnect. (This is what I called noXBI in the previous <a href="https://joinmarket.me/blog/blog/thehalfscriptlessswap/">blog post</a>).</p>
<p><strong>Boundary</strong>  the adversary A, as was hinted at in the introduction, in this model, will not necessarily be able to easily see on the blockchain where the start and end points of this flow of transactions was. To the extent that this is true, it's an enormous win, but more on this later.</p>
<h2>Example</h2>
<p><img alt="ExampleCJXT" height="422" src="https://joinmarket.me/static/media/uploads/.thumbnails/onchaincontract3.png/onchaincontract3614x422.png" width="614"/></p>
<p>Here we are still restricting to 2 parties for simplicity of the diagram. There is still a chain of 4 TXs, but here we flesh out the inputs and outputs. About colors:</p>
<p>Blue txos are coowned by the two parties, envisioned as 2 of 2 multisig (although as originally mentioned, the technical requirement is only that each transaction is signed by both parties).</p>
<p>Red inputs are <strong>promise utxos</strong> as described in the earlier section.</p>
<p>Each promise has a corresponding backout transaction presigned as output consuming the bitcoins of the <span style="textdecoration: underline;">previous</span> transaction to the one consuming that promise.</p>
<p>Notice that this example contains two possible setups for each individual transaction in the chain; it can pay out only to one party (like TX3 which pays bob 0.6btc), or it can pay "CoinJoinstyle" equalsized outputs to 2 (or N) parties. Choosing this latter option means you are consciously deciding to blur the line between the <em>intrinsicfungibility </em>model and the <em>deniability</em> <em>model, </em>which, by the way, is not necessarily a bad idea.</p>
<h2>The return of A  amounts leak.</h2>
<p>As mentioned, our adversary A has a very important problem  he may not know that the above negotiation has happened, unlike a simple CoinJoin where the transactions are watermarked as such (and this is particularly true if Alice and Bob do <em>not</em> use equalsized outputs). The boundary may be unclear to A.</p>
<p>So, what strategy <em>can</em> A use to find the transaction graph/set? He can do <a href="https://en.wikipedia.org/wiki/Subset_sum_problem">subset sum</a> analysis.</p>
<p>If Alice and Bob are just 'mixing' coins, so that they are paid out the same amount that they paid in, I'll assert that subset sum is likely to work. It's true that A's job is quite hard, since in general, he would have to do such subsetsum analysis on a huge array of different possible sets of (inputs, outputs) on chain; but nevertheless it's the kind of thing that can be done by a professional adversary, over time. The fact that subset sum analysis is theoretically exponential time and therefore not feasible for very large sets may not be relevant in practice.</p>
<p>In our example above it may not be hard to identify the two inputs from Alice (1btc, 0.3btc) as corresponding to 3 outputs (0.8btc, 0.2btc, 0.3btc), albeit that the latter two  0.2, 0.3 were part of CoinJoins. Remember that this was a tradeoff  if we <em>didn't</em> make equal sized outputs, to improve deniability/hiding, we'd no longer have any ambiguity there.</p>
<h2>Breaking subsetsum with Lightning</h2>
<p><img alt="" height="392" src="https://joinmarket.me/static/media/uploads/.thumbnails/amtdecorr2.png/amtdecorr2711x392.png" width="711"/></p>
<p>Here's one way of addressing the fact that A can do subsetsum on such a privacyenhancing CoinJoinXT instantiation. The PTG is unspecified but you can imagine it as something similar to the previous example.</p>
<p>Marked in blue is what the adversary A doesn't know, even if he has identified the specific transaction/graph set (as we've said, that in itself is already hard). Subsetsum analysis won't work here to identify which output belongs to Alice and which to Bob; since 5.5 + 1.5 != 6.6, nor does 5.4 fit, nor does such an equation fit with Alice's input 5.8 on the right hand side of the equation.</p>
<p>The trick is that the 1.5 output is actually a <strong>dual funded Lightning channel</strong> between Alice and Bob. The actual channel balance is shown in blue again because hidden from A: (0.3, 1.2). If the channel is then immediately closed we have fallen back to a case where subset sum works, as the reader can easily verify.</p>
<p>But if, as is usually the intent, the channel gets used, the balance will shift over time, due to payments over HTLC hops to other participants in the Lightning network. This will mean that the final closing balance of the channel will be something else; for example, (0.1, 1.4), and then subsetsum will still not reveal which of the 2 outputs (5.4, 5.5) belong to Alice or Bob.</p>
<p>At a high level, you can understand this as a <strong>bleedthrough and amplification of offchain privacy to onchain.</strong></p>
<p>It's worth noting that you clearly get a significant part of this effect from just the dualfunded Lightning channel; if you consider change outputs in such a single funding transaction, you see the same effect:</p>
<pre>Alice<br/>2.46<br/> > Lightning funding 0.1<br/> > Change 2.41<br/> > Change 2.37<br/>2.42<br/>Bob</pre>
<p>It's easy to see that there is no delinking effect on the changeouts <em>if</em> we know that the funding is equal on both sides. However, there's no need for that to be the case; if the initial channel balance is (Alice: 0.09, Bob: 0.01) then the changeouts are going to the opposite parties compared to if the channel funding is (Alice: 0.05, Bob: 0.05). So this concrete example should help you to understand a crucial aspect of this:</p>
<ul>
<li>Such a fungibility effect is only achieved if the difference between the two parties' initial inputs is small enough compared to the size of the dualfunded Lightning channel</li>
<li>If the size of the inputs is very large compared to the Lightning channel overall size, which currently at maximum is 2**24 satoshis (about 0.16btc), then, in order to achieve this obfuscation effect, we "converge" to the case of something like a 2in and 2out equalsized coinjoin. It's hard for 2 parties to arrange to have inputs of equal sizes, and it somewhat loses the deniability feature we were going for. (You can easily confirm for yourself that there will be no ambiguity if Alice and Bob's inputs are of completely different sizes).</li>
</ul>
<p>So how does the picture change if instead of just doing a single dualfunded Lightning channel, we include it as an output in a CoinJoinXT structure?</p>
<p>The answer again is deniability. Any contiguous subset of the entire blockchain has the property of sum preservation, modulo fees: the input total is ~= the output total. So no particular contiguous subset on the blockchain flags itself as being such a CoinJoinXT structure  unless subset sum works for some N subsets (2, as in our examples, or higher). But with the dual funded Lightning output of the type shown here, at least for the 2 of 2 case, this doesn't work.</p>
<p></p>
<h2>Remove all traces?</h2>
<p>What's been described up to now doesn't quite achieve the desired goal of "deniability"; there are still what we might call "fingerprints" in such a CoinJoinXT structure:</p>
<ul>
<li>Timing correlation: if we don't use nLockTime on these transactions, then one party might choose to broadcast them all at once. This is at the least a big clue, although not unambiguous. To avoid it, have the presigned transactions in the PTG all be given specific timelocks.</li>
<li>Shared control utxos. If we use 2 of 2, or N of N, multisig outputs, of the current normal p2sh type, then they are observable as such, and this could easily help A to find the "skeleton" of such a CoinJoinXT structure. Of course, let's not forget that we can do CoinJoinXT with various equal sized outputs too, mixing the "intrinsic fungibility" and "deniability" approaches together, as discussed, so it's not that CoinJoinXT with p2sh multisig connecting utxos is useless. But we may want to focus on less detectable forms, like Schnorr/MuSig based multisig with key aggregation so that N of N is indistinguishable from 1 of 1, or the new <a href="https://eprint.iacr.org/2018/472">construction</a> that allows an ECDSA pubkey to be effectively a 2 of 2 multisig.</li>
</ul>
<p></p>
<h2>Conclusion</h2>
<p><strong>Proof of Concept </strong> I put together a some very simple <a href="https://github.com/AdamISZ/CoinJoinXTPOC">PoC code</a>; it only covers something like the above first "Example" with 2 parties. Going through such an exercise in practice at least allows one to see concretely that (a) the interaction between the parties is very minimal (subsecond) which is great of course, but it gets a little hairy when you think about how to set up a template of such a transaction chain that 2 parties can agree on using whatever utxos they have available as inputs. A substantial chunk of that PoC code was devoted to that  there is a general <code>Template</code> class for specifying a graph of transactions, with parametrized input/output sizes.</p>
<p><strong>Practicality today</strong>  Although it can be done today (see previous), there are barriers to making this work well. Ideally we'd have Schnorr key aggregation for multisig, and support for dual funded Lightning channels for the amount decorrelation trick mentioned. Without either of those, such a transaction graph on the blockchain will be <em>somewhat</em> identifiable, but I still think there can be a lot of use doing it as an alternative to large sets of clearly identifiable CoinJoins.</p>
<p><strong>Cost tradeoffs </strong> left open here is the tradeoffs in terms of blockchain space usage for each "unit of fungibility", i.e. how much it costs to gain privacy/fungibility this way. I think it's almost impossible to come up with definitive mathematical models of such things, but my feeling is that, exactly to the extent any "deniability" is achieved, it's costeffective, and to the extent it's not, it's not costeffective.</p>
<p><strong>Coordination model </strong> Currently we have "in play" at least two models of coordination for CoinJoin  Joinmarket's marketbased model, and the Chaumian server model currently championed by <a href="https://github.com/nopara73/ZeroLink">ZeroLink</a>. <strong>CoinJoinXT as an idea is orthogonal to the coordination mechanism</strong>. The only "nonorthogonal" aspect, perhaps, is that I think the CoinJoinXT approach may still be pretty useful with only 2 parties (or 3), more so that CoinJoin with only 2/3.</p>
<p>Finally, where should this fit in one's fungibility "toolchest"? Lightning is <em>hopefully</em> going to emerge as a principal way that people gain fungibility for their everyday payments. The area it can't help with now, and probably not in the future due to its properties, is with larger amounts of money. So you might naturally want to ensure that in, say, sending funds to an exchange, making a largeish payment, or perhaps funding a channel, you don't reveal the size of your cold storage wallet. I would see the technique described on this blog post as fitting into that mediumlarge sized funds transfer situation. CoinJoin of the pure "intrinsic fungibility" type, done in repeated rounds or at least in very large anonymity sets, is the other alternative (and perhaps the best) for large sizes.</p>The half scriptless swap20180625T15:07:27+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/thehalfscriptlessswap/<h1>A curious, hybrid, unlinkable, signature algorithm independent atomic swap</h1>
<p>(<strong>**THIS ALGORITHM IS BROKEN** .. OOPS! LEAVING FOR POSTERITY, BUT I HAVE MARKED WITH *** A COMMENT IN THE BELOW THAT IDENTIFIES THE FLAW IN THE REASONING. Also, there is a way I think it could be made to work, but only in a more restricted context than initially envisioned; again, see the comment below marked with ***).</strong></p>
<p><strong></strong></p>
<p>(<strong>Note as of 30 Jun</strong>: edited to alter the algorithm steps; note there is more than one way to do this.)</p>
<p>Previous blog posts (<a href="https://joinmarket.me/blog/blog/feeds/atom/%22https://joinmarket.me/blog/blog/coinswaps/">here</a>, <a href="https://joinmarket.me/blog/blog/flippingthescriptlessscriptonschnorr/">here</a>) covered the idea of atomic swap, how to make it private/unlinkable (at least under cooperation), and how to get a much better version after the advent of Schnorr signatures (the "scriptless script" primitive of Poelstra).</p>
<p>In this blog post I want to show <span style="textdecoration: underline;">one</span> of a number of possible variations. It is "hybrid" only in a loose sense: there are a lot of previously existent ideas that I've here mixed together to form this particular set of properties.</p>
<blockquote><strong>The TLDR is: an atomic swap (one blockchain or crosschain) with *no* linkage between the transactions, even if one party chooses to be uncooperative, and not dependent on the two chains using the same signature algorithm.</strong></blockquote>
<p>Before starting, it's worthy of note that I may be slightly abusing the term "unlinkable" here; the swap described is not unlinkable by the participants, but only by third party observers. Achieving the former is possible: see Tumblebit (linked below), but also a recent <a href="https://github.com/jonasnick/scriptlessscripts/blob/blindswaps/md/partiallyblindswap.md">proposal</a> by Jonas Nick, which uses Schnorr blind signatures.</p>
<p>To go into the details, we first need some background for context:</p>
<p></p>
<h2>Weaknesses of multitransaction contracts</h2>
<p>For the purposes of this discussion, it's quite useful to start (assuming you're already au fait with what an atomic swap basically <em>is</em>, and/or have read the above two blog posts) by noting what the limitations and problems are with doing atomic swaps (not shared by all variants). They are:</p>
<ol>
<li>Interactivity  (this is also shared by CoinJoin, except see <a href="https://joinmarket.me/blog/blog/snicker/">SNICKER</a> (which doesn't apply here))</li>
<li>XBI  A term I just coined, meaning "crossblockinteractivity"  I consider this a separate and far more important limitation than the previous. For two parties to simply share some data over a network is a bit of an issue, but a <strong>far</strong> bigger issue, in my opinion, is having multiple rounds of interactivity <strong>separated by waiting for confirmations on the blockchain</strong>. This somewhat overlaps with the next issue:</li>
<li>Security assumptions about the blockchain  any contract that involves more than one <em>disconnected </em>transaction must be protected from deadlock with timed backout paths (separate txs or paths in script); which means that the inherent miner censorship risk goes from "I may not be able to spend for a while until Jihan gets bored" to "I may lose <strong>all</strong> the money in this contract if a cartel of miners decide they don't like me for <em>a few hours</em>" or, far more likely "<em>if the internet is blocked for a few hours in the Republic of Bitcoin where I live</em>". The "a few hours" is, of course, configurable, but with a usability tradeoff. In short, there's a <strong>liveness assumption </strong>and a <strong>noncensorship assumption</strong>, with the former being the really important one.</li>
<li>Privacy  the naive atomic swap construct in Script (using hash preimages) has no privacy, necessarily (due to entropy requirement of hash preimage), from any even slightly competent snooper. On the other hand, the best (afaik) variant of atomic swap/CoinSwap we have today, somewhat confusingly called "scriptless script atomic swap" (see second of two blog posts), completely solves this issue (even in noncooperation) as well as having other advantages. It can't be done today with Schnorr signatures on Bitcoin, because we don't have them; it can be done according to the recent work of MorenoSanchez et al using ECDSA 2PC, but this is relatively new (which I may talk about in a future post).</li>
<li>Compatibility  both transactions ideally occur on the same blockchain, or if different, they must be compatible in having either the same signature algorithm and perhaps elliptic curve (thinking scriptless scripts), or the same hash function in their Script (if they even have Script).</li>
</ol>
<p>You'll note I've been talking about "atomic swaps", but that the subtitle heading refers to "multitransaction contracts". These concerns all apply to Lightning network which in its "network" part currently uses HTLC hops (with HTLC being the basic atomic swap) and in its "Lightning" part uses the <a href="https://lightning.network/lightningnetworkpaper.pdf">PoonDryja bidirectional channel construct</a>, which also uses timing control as a crucial element of its security promises.</p>
<p>I hope it's clear why some combination of point 2 (XBI) and most critically point 3, above, (security assumptions about the blockchain itself) are a very big issue for all of these constructions. If you're planning to CoinSwap your life savings (or Lightningswap/route through channels), you'd better be sure your internet is working, your blockchain access is viable, and (far fetched, but..) the miners aren't out to get you. To be fair, the Lightning people are working on <a href="https://diyhpl.us/wiki/transcripts/scalingbitcoin/milan/unlinkableoutsourcedchannelmonitoring/">watchtowers</a>, but that's an amelioration, not a solution, to this inherent problem.</p>
<p>Lastly, in this section, I want to point out a small nuance which will become important in my next blog post: these concerns don't apply to a certain class of multitransaction contracts which are connected (i.e. a tree structure with a single root). But that's another topic and not relevant here.</p>
<p></p>
<h2>Motivation, and previous related ideas</h2>
<p>The new flavour of swap I'm going to propose addresses points 4 and 5 in the above, and <em>partially</em> point 2 ("XBI"), if we allow that one half of the swap is identifiably a nonstandard transaction (revealing hash preimage). You can increase privacy by adding more interactivity, but what I present from here is a version with interactivity minimized.</p>
<p>So, since it's rather a niche, and it doesn't address point 3 and only partially point 2, I think this idea is interesting but not exactly amazing :)</p>
<p>In thinking about how to try to solve some of these issues, I remembered in particular the mechanics of Tumblebit (my <a href="https://joinmarket.me/blog/blog/tumblebitforthetumblecurious/">blog post</a>, <a href="https://eprint.iacr.org/2016/575.pdf">original paper</a>), and also the general idea of Greg Maxwell called <a href="https://en.bitcoin.it/wiki/Zero_Knowledge_Contingent_Payment">Zero Knowledge Contingent Payments</a> from 2011.</p>
<p>In recent discussion Olaoluwa Osuntokun also pointed me at <a href="https://eprint.iacr.org/2016/451.pdf">this</a> paper from 2015/2016 by University of Warsaw researchers Banasik, Dziembowski and Malinowski, which actually is discussing a lot of closely related ideas: how can we effect something like zero knowledge contingent payment using cut and choose, but it also discusses ways of creating single ECDSA pubkeys through 2PC and the Paillier encryption system. It also uses a number of other elements which I decided weren't needed for my goal as described here.</p>
<p>Tumblebit involves two cutandchoose protocols, with a blinding step between them to prevent the server learning a linkage. But the root idea of the first protocol, called the "Puzzle Promise Protocol", struck me as quite generic: I will give you an encryption of a signature and a hash of its decryption key. If you can learn the hash preimage, you can learn the signature and spend the coin (basically). This cut and choose is a very generic kind of interactive zero knowledge proof, although it has a quirk  to gain reasonable security, you need a whole set of valid signatures, not just one; in the Bitcoin transaction scenario, however, that doesn't matter, because the entire point of the blockchain is to prevent doublespending.</p>
<p>(Of course, the ZKCP construction is basically doing the same thing, just more generically/powerfully, and doesn't require provision of multiple puzzle solutions simultaneously, and importantly, <strong>it's noninteractive</strong>: you can encrypt a solution to a puzzle and prove in zero knowledge using systems like zkSNARKs that your solution is valid and then sell the preimage of the decryption key, as above. These more powerful proving systems, though, have bigger computational requirements, see the first example <a href="https://bitcoincore.org/en/2016/02/26/zeroknowledgecontingentpaymentsannouncement/">ZKCP</a>, in which 20s was recorded as the proving time; note this has almost certainly improved, although Sudoku is kind of a toy example. Also, I believe the example shown there was proven to have invalid assumptions, specifically that the payer can be trusted to generate the CRS, but it turned out that this broke the witness indistinguishability property that guaranteed the zero knowledgeness property of the proof; see <a href="https://eprint.iacr.org/2017/566.pdf">this paper</a> for details and fixes. But  tangent over! ))</p>
<p>So what if we do such a cutandchoose, and then have the hash preimage revealed in a separate, disconnected transaction? There are two advantages of doing things this way: (1) no linkage is possible because the hash preimage is only revealed in one transaction (and <em>only</em> in the noncooperative case), and (2) the exact signature algorithm of the first payment (the one whose sig is encrypted) is irrelevant. This might mean, for example, that doing a swap between Bitcoin and a blockchain which uses a different elliptic curve for its signatures (e.g. <a href="https://en.wikipedia.org/wiki/Curve25519">curve25519</a>) and/or a different signature scheme (<a href="https://en.wikipedia.org/wiki/EdDSA">edDSA</a>, which is a variant of Schnorr, for example, or perhaps less well known things like <a href="https://en.wikipedia.org/wiki/Merkle_signature_scheme">hashbased signatures</a>).</p>
<p></p>
<p>The next 3 subsections will discuss the steps in what has just been outlined very briefly. We'll revisit our friends Alice and Bob; we'll assume Bob's 1 coin resides on the BTC blockchain in utxo UB and we'll say Alice's 1 Alicecoin is on the Alicechain (not specifying which blockchain it is, and magically assuming that 1 Alice coin = 1 BTC in value) in utxo UA.</p>
<p>M(N, M, key1, key2, ...) refers to a N of M multisig on those keys.</p>
<h2>First stage  the interaction</h2>
<ol>
<li>CONNECT</li>
<li>Negotiate keys: A1,A2, A3, B1. Create M(2,2,A1,B1). Alice create txid for TX1 from UA to M(2,2,A1,B1), give to Bob. Bob presign backout to A2 with nlocktime L1.</li>
<li>Alice gives key A3 as ephemeral key for BTC side of swap.</li>
<li>A and B do "cut and choose" (see below), ending with Bob receiving \(H(k)\) and \(E_k(\sigma)\) for M distinct cases, where \(\sigma_M\) is a signature on M(2,2,A1,B1)>B2_M.</li>
<li>DISCONNECT</li>
</ol>
<p>First some commentary on the design before explaining in detail the meaning of Cut and Choose here (the next subsection, below).</p>
<p>You'll recall from the "CoinSwaps" blogpost that a simple atomic swap allows the participants to simply pay into custom scriptPubKeys of type "HTLC" (hash, key1 or timeout, key2), which removes XBI at the price of having hash preimages linked across the txs. The alternative is to have the two parties pay into a shared control outpoint  i.e.e a 2 of 2 multisig  this allows use of "overlay" payout transactions which don't involve hash preimages and thus preserve privacy. This you get at the cost of XBI since you have to wait for the funding 2 of 2s to confirm (as in Lightning).</p>
<p>So, here, what we are doing is to remove the XBI by not requiring Alice, the holder of the secret, to pass it across to Bob after this 1 phase of interactivity (after any confirmations), like in the basic atomic swap, but we won't lose the privacy since the hash preimage will only be revealed one side. This will be clear after the "Second stage" is written out, hopefully. You'll note that we have tacitly assumed that Alicechain supports 2 of 2 multisig in some or other form, and nLockTime or something directly equivalent.</p>
<h2>Cutandchoose</h2>
<p>This is a brief refresher that is basically identical to what was called the "Puzzle Promise protocol" in Tumblebit (for links see above). Since it's the core idea here, it's worth going over the steps:</p>
<ul>
<li>Bob prepares \(N\) fake hashes of format \(H(\textbf{fixedstringforallfakes}y)\) where \(y\) is random, for blinding.</li>
<li>Bob prepares \(M\) real transactions paying from M(2,2,A1,B1)>B2_M to his own destination addresses. Each destination must be different (he does not communicate these addresses of course).</li>
<li>Bob commits to a jumbled order of the \(N+M\) real and fake hashes and sends that commitment along with the hashes themselves (which are all, also, commitments  both real and fake have the hiding and binding property)</li>
<li>Alice signs all of the hashes, not knowing the difference. The signing algorithm on Alicechain, remember, is unspecified. Then she chooses independent symmetric encryption keys \(k_i\) for each of them, and encrypts each of them to \(E_i\) with those keys (using e.g. xor). She sends to Bob \((H(k_i), E_i) \ \forall i\)</li>
<li>Bob sends to Alice the openings of the \(N\) fake hashes; Alice verifies correct preimage format, then passes to Bob the decryption keys for the fakes.</li>
<li>Bob verifies that the preimages of \(H(k_i)\) values match the revealed \(k_i\)s, and that the fake hashes were properly signed using the appropriate verification function for the signature algorithm on Alicechain (digital signatures must all have at least the three algorithms KeyGen, Sign and Verify).</li>
<li>Now Bob knows that each of the remaining \(H(k_i)\) values are in fact hashes of the correct decryption keys for the corresponding \(E_i\)s with security \(1/\binom{N+M}{M}\). This can be about 80 bit security with \(N=M=42\), from a simple combinatorial argument, as observed in the Tumblebit paper and blog. <strong>***THIS REASONING IS FLAWED: Tumblebit used the RSA quotient test to ensure a vital property: that if any one of the M remaining decryption keys is valid, so will they all be; without that property, and allowing successful decryption with any one of them, we don't get the required combinatorial argument; to succeed in cheating you just need to have guessed which specific "real" decryption key hash is the one that's going to get picked. So it's linear rather than combinatorial, which is far too weak for realistic N, M. If we wanted to keep this property without RSA we'd need to use a similar homomorphism; I think if we replace the hashes of keys with curve points (like K = kG, with k the decryption key), it might be possible to make the construction work; but this would probably mean assuming the use of something like Schnorr, which we were trying to avoid. I haven't yet tried to work out the details.).</strong></li>
</ul>
<p>This process requires a bit of interactivity of course; but it's not XBI and it's not crazy large (with fast network connections, this basically 2 rounds of interactivity could be effectively immediate, which <em>may</em> (although it's a stretch) be considered an advantage over a noninteractive protocol like zkSNARKs which may take an appreciable amount of resources/time.</p>
<p></p>
<h2>Second stage  the broadcasts</h2>
<p>The remaining steps of the protocol can be done without Alice and Bob communicating, but each side's security does critically depend on timely access to the blockchain.</p>
<p>These steps must be in the following "serial" order, not done in parallel, for reasons that'll be clear if you think about it.</p>
<ol>
<li>A broadcasts the transaction TX1: UA>M(2,2,A1,B1). <strong>Bob must wait for this to confirm</strong>.</li>
<li>Bob now prepares a transaction TX2 paying from UB to an output whose scriptPubKey is roughly:</li>
</ol>
<pre>IF HASH H(k_i) EQV A3 CHECKSIG <br/>ELSE L2 CHECKLOCKTIMEVERIFY DROP B3 CHECKSIG ENDIF</pre>
<p>where i is just one of the 42 realhash indices at random. Bob broadcasts this. <strong>Alice must wait for this to confirm.</strong></p>
<p>After it confirms, Alice can now, at any time before L2, broadcast a transaction spending from this TX2, claiming her 1BTC and revealing \(x\). Bob will then decrypt \(E_k(\sigma)\) and retrieve his 1 Alice coin.</p>
<p>A final note (common to all these designs): the locktime L2 must be closer in time than L1 so that Alice can't wait until after L1 to backout her commitment and claim the 1BTC with \(H(x)\).</p>
<h2>Summary</h2>
<p>As was mentioned at the beginning, this is not a <em>very</em> interesting alternative to existing atomic swap constructions, but I wanted to write it out for a few reasons:</p>
<ol>
<li>It does not involve any correlation between the two payments, whatever blockchain they're on, even in the case of noncooperation (here, Alice has to broadcast the transaction paying to a hash preimage). This is because the cutandchoose "side" of the protocol doesn't have any information in its redeeming step; just a spend of a multisig output, whether it was cooperative or not. This is better than the Maxwellstyle CoinSwap or the original atomic swap based on hash preimages on both sides (which therefore correlate).</li>
<li>There are corner cases where it solves a problem that even scriptless scripts couldn't  specifically with incompatible blockchains, different curves, different signature algos, or perhaps the lack of a hash scripting facility (although it needs both multisig and some kind of timelock, so not sure how many such blockchains exist!)</li>
<li>You can certainly write a different version of this protocol (as I did originally before editing!  where you avoid publishing a hash preimage in the cooperative case; as mentioned, the tradeoff is you put in more "XBI" which I think is very undesirable).</li>
<li>Perhaps most importantly, I wanted to write this out so that it might give other people ideas about how to use this toolset  cut and choose, interactive ZKP, multi transaction contracts, in other interesting ways. I'm particularly interested to know if people have ideas about how to make the interactivity less, but it might not be possible if your goal is specifically to swap coins into different histories.</li>
</ol>The steganographic principle20180301T16:54:56+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/thesteganographicprinciple/<h1>The Steganographic Principle</h1>
<p>Some time ago I wrote <a href="https://gist.github.com/AdamISZ/83a17befd84992a7ad74">this</a> gist, which is an illformed technical concept about a way you could do steganography leveraging randomness in existing network protocols; but I also called it a "manifesto", jokingly, because I realised the thinking behind it is inherently political.</p>
<h2>Cryptography is for terrorists, too</h2>
<p>There are a few reasons why the phrase "If you have nothing to hide, you have nothing to fear" is wrong and insidiously so. One of the main ones is simply this: my threat model is <strong>not only my government</strong>, even if my government is perfect and totally legitimate (to me). But no government is perfect, and some of them are literally monstrous.</p>
<p>So while it's true that there are uses of cryptography harmonious with a PG13 version of the world  simply protecting obviously sensitive data <em>within</em> the control of authorities  there are plenty where it is entirely ethically right and necessary to make that protection <strong>absolute</strong>.</p>
<p>The question then arises, as was raised in the above gist, what are the properties of algorithms that satisfy the requirement of defence even against hostile authorities?</p>
<p>The modern tradition of cryptography uses Kerckhoff's Law as one of its axioms, and steganography does not fit into this model. But that's because the tradition is built by people in industry who are fine with people <strong>knowing they are using cryptography</strong>. In an environment where that is not acceptable, steganography is not on a list of options  it's more like the sine qua non.</p>
<h2>Steganography on blockchains</h2>
<p>On a blockchain, we have already understood this "freedom fighter" model. It's an essential part of how the thing was even created, and why it exists. And there are essentially two principal complaints about Bitcoin and its blockchain, both of which are somewhat related to this:</p>
<ul>
<li>Privacy</li>
<li>Scalability</li>
</ul>
<p>The first is obvious  if we don't create "steganographic" transactions, then governments, and everyone else, may get to know at least <em>something</em> about our transactions. The second is less so  but in the absence of scale we have a small anonymity set. Smaller payment network effects and smaller anonymity sets obviously hamper use of these systems by a "freedom fighter". But remember the scale limitations come directly out of the design of the system with censorship resistance and independent verification in mind.</p>
<p>Attempts to improve the privacy by altering the <em>way</em> in which transactions are done have a tendency to make the scalability worse  the obvious example being CoinJoin, which with unblinded amounts inevitably involves larger numbers of outputs and larger numbers of transactions even.</p>
<p>A less obvious example is Confidential Transcations; when we blind outputs we need to use up more space to create the necessary guarantees about the properties of the amounts  see the range proof, which with Borromean ring signatures or bulletproofs need a lot of extra space. The same is true of ring signature approaches generally to confidentiality.</p>
<p>You can trade off space usage for computation though  e.g. zkSNARKs which are quite compact in space but take a lot of CPU time to create (and in a way they take a lot of space in a different sense  memory usage for proof creation).</p>
<h2>Localised trust</h2>
<p>You can improve this situation by localising trust in space or time. There are obvious models  the bank of the type set up by digicash. See the concept of <a href="https://en.wikipedia.org/wiki/Blind_signature">Chaumian tokens</a> generally. One project that looked into creating such things was <a href="https://github.com/OpenTransactions/">OpenTransactions</a>, another was Loom, also see Truledger.</p>
<p>Trust can be localised in time as well  and the aforementioned zkSnarks are an example; they use a trusted setup as a bootstrap. This trust can be ameliorated with a multiparty computation protocol such that trust is reduced by requiring all participants to be corrupt for the final result to be corrupt; but it is still trust.</p>
<h2>The tension between privacy and security</h2>
<p>For any attribute which is perfectly (or computationally) hidden, we have a corresponding security downgrade. If attribute A is required to satisfy condition C by the rules of protocol P, and attribute A is blinded to A* by a privacy mechanism M, in such a way that we use the fact that C* is guaranteed by A*, then we can say that P's security is "downgraded" by M in the specific sense that the Cguarantee has been changed to the C*guarantee, where (inevitably) the C* guarantee is not as strong, since it requires the soundess of M as well as whatever assumptions already existed for the soundness of C.</p>
<p>However, the situation is worse  precisely because M is a privacy mechanism, it reduces public verifiability, and specifically verifiability of the condition C, meaning that if the C* guarantee (which we <em>can</em> publically verify) fails to provide C, there will be no public knowledge of that failure.</p>
<p>To give a concrete example of the above template, consider what happens to Bitcoin under Confidential Transactions with Pedersen commitments (set aside the range proof for a moment). Since Pedersen commitments are perfectly hiding but only computationally binding, we have:</p>
<p>P = Bitcoin</p>
<p>A = Bitcoin amounts of outputs</p>
<p>C = amount balance in transactions</p>
<p>M = CT with Pedersen commitments</p>
<p>A* = Pedersen commitments of outputs</p>
<p>C* = Pedersen commitment balance in transactions</p>
<p>Here the downgrade in security is specifically the computational binding of Pedersen commitments (note: that's assuming both ECDLP intractability *and* NUMSness of a curve point). Without Pedersen/CT, there are *no* assumptions about amount balance, since integers are "perfectly binding" :) With it, any failure of the computational binding is catastrophic, since we won't see it.</p>
<h2>The tension between privacy and scalability</h2>
<p>For any attribute A which is obfuscated by a privacy mechanism M in protocol P (note: I'm choosing the word "obfuscation" here to indicate that the hiding is not perfect  note the contrast with the previous section), we have a corresponding scalability failure. M may obfuscate an attribute A by expanding the set of possible values/states from A to A[N]. To commit to the obfuscation soundly it must publish data of order ~ N x size(A). Also note that it is <em>possible</em> for the obfuscation goal to be achieved without an increase in space usage, if multiple parties can coordinate their transactions, but here we ignore this possibility because it requires all parties to agree that all attributes except A to be identical (example: multiple participants must accept their newly created outputs are equal value). This is not really a "transaction" in the normal sense.</p>
<p>A concrete example: equalsized Coinjoin in Bitcoin:</p>
<p>P = Bitcoin</p>
<p>A = receiver of funds in a transaction</p>
<p>A[N] = set of N outputs of equal size</p>
<p>M = Coinjoin</p>
<p>A less obvious example but fitting the same pattern; ElGamal commitment based Confidential Transactions (as opposed to Pedersen commitments based)</p>
<p>P = Bitcoin</p>
<p>A = output amount in a transaction</p>
<p>A[N] = ElGamal commitment to amount, here 2 curve points, N=2</p>
<p>M = ElGamal commitments</p>
<p>Here N=2 requires some explaining. An ElGamal commitment is perfectly binding, and to achieve that goal the commitment must have 2 points, as the input has two values (scalars), one for blinding and the other for binding the amount. So we see in this case the expansion in practice is more than just a single integer, it's from a single bitcoinencoded integer to two curve points. But the details obviously vary; the general concept is to whatever extent we obfuscate, without throwing in extra security assumptions, we require more data.</p>
<h2>Verification  public or private?</h2>
<p>The structure above is trying to make an argument, which I believe is pretty strong  that this represents searching for privacy, in a blockchain context, in slightly the wrong way.</p>
<p>If we try to make the <em>blockchain </em>itself private, we are slightly pushing against its inherent nature. Its crucial feature is <span style="textdecoration: underline;">public verifiability</span>, and while it's true that this does not require all attributes properties to be "unblinded" nor "unobfuscated", we see above that introducing blinding or obfuscation is problematic; you either degrade security in a way that's not acceptable because it introduces invisible breaks, or you degrade scalability (such as using a perfectly binding commitment requiring no compression, or a zero knowledge proof taking up a lot of space or computation time), or you degrade trustlessness (see: trusted setup zkps). I have no absolute theorem that says that you cannot get rid of all of these problems simultaneously; but it certainly seems hard!</p>
<p>This is where the idea of a "steganographic blockchain" comes in; if instead of trying to hide attributes of transactions, we try to make the <em>meaning</em> of transactions be something not explicit to the chain, but agreed upon by arbitrary participants using mechanisms outside it. This allows one to leverage the blockchain's principal feature  censorship resistant proof of state changes, in public, without inheriting its main bugs  lack of privacy and scalability, and without degrading its own security.</p>
<p></p>
<p>Examples:</p>
<ul>
<li>Colored coins</li>
<li>Crude example: atomic swaps</li>
<li>Lightning and secondlayer</li>
<li>Chaumian tokens</li>
<li>Clientside validation (single use seals)</li>
<li>Scriptless scripts</li>
</ul>
<h2>High bandwidth steganography</h2>
<p>The biggest practical problem with steganography has always been bandwidth; if you use nonrandom data such as images or videos, which are often using compression algorithms to maximise their signal to noise ratio, you have the problem of getting sufficient "cover traffic" over your hidden message.</p>
<p>Note that this problem does not occur <strong>at all</strong> in cases where your hidden message is embedded into another message which is random. This is the case with digital signatures; ECDSA and Schnorr for example are both publish as two random values each of which is about 32 bytes.</p>
<p>To go back to the previously mentioned example of scriptless scripts, we can see that the atomic swap protocol based on it as described in my <a href="https://joinmarket.me/blog/blog/flippingthescriptlessscriptonschnorr/">blog post</a>, exploits this directly. On chain we see two (not obviously related) transactions with Schnorr signatures that are, to the outside observer, in no way related; the hiding of the connection is perfect, but the binding/atomicity of the two payments is still secure, just not perfectly so (it's based on the ECDLP hardness assumption, but then so are ordinary payments).</p>
<p>Note how this is a different philosophy/approach to hiding/privacy: since such a swap leaves no fingerprint onchain, the concept of anonymity set blurs; it's strictly all transactions (assuming Schnorr in future, or ECDSA2PC now), even if most people do not use the technique. To get that same effect with an enforced privacy overlay mechanism M for all participants, we tradeoff the security or scalability issues mentioned above.</p>
<p>This is the reason for my slightly clickbaityy subtitle "High Bandwidth Steganography". A big chunk of the Bitcoin blockchain is random (as those who've tried to compress it have learned to their chagrin), and so it's not quite as hard to usual to hide transaction semantics (the ideal case will be inside signatures using scriptless script type constructs), so in a sense we can get a very high bandwidth of data communicated client to client without using any extra space on chain, and without "polluting" the chain with extra security assumptions.</p>
<p></p>From Zero Knowledge Proofs to Bulletproofs Paper20180226T15:20:15+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/fromzeroknowledgeproofstobulletproofspaper/<p>I've spent the last few weeks working on this paper, which comes out of my own desire to understand the technical underpinnings of Bulletproofs (see my previous post). It ends up being a walkthrough of sections of three academic papers, with "Asides" along the ways about various supporting concepts like Commitments and Zero Knowledge Proofs.</p>
<p>This is to be considered just a first version, I intend to move it to github (once I've figured out some format conversion issues), and I'm very happy to receive some comments and corrections from anyone out there so inclined (message me on IRC or twitter).</p>
<p>Thanks to Andrew Poelstra for helping me get started in understanding this (it's kind of intimidating/overwhelming at first, part of why I wrote the doc), and also to Jonas Nick who I discussed some details back and forth with. And of course thanks to all the authors of the three papers under discussion :)</p>
<p><strong>Edit: </strong>Now hosting this in .tex and pdf at <a href="https://github.com/AdamISZ/from0k2bp">https://github.com/AdamISZ/from0k2bp</a> ; forgive some glitches here and there, updates will occur as and when; please open an issue if you have a correction, question or comment on the paper. Thank!</p>
<p><strong>For historical reference, first version of the paper:</strong></p>
<p><a href="https://joinmarket.me/static/FromZK2BPs_v1.pdf">Link to Version 1 pdf</a></p>Bulletpoints on bulletproofs20171122T15:22:19+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/bulletpointsonbulletproofs/<h2>Bulletproofs</h2>
<p>The recent publication of <a href="https://eprint.iacr.org/2017/1066">this</a> paper by Benedikt Bünz of Stanford (and coauthors) has caused quite a stir amongst Bitcoin people. It's kind of obscure for a non or semi technical audience, but most people who heard about it got the gist: this may be the way to finally make Confidential Transactions (transactions that hide the amounts) practical for Bitcoin.</p>
<p>If you just want the <em>effects</em> of this on a potential future BitcoinorsidechainwithCT, Greg Maxwell has you covered <a href="https://lists.linuxfoundation.org/pipermail/bitcoindev/2017November/015283.html">here</a>. What I write below is for those curious and interested at a more technical/mathematical level (but read Greg's post anyway!).</p>
<p>So in this post I want to go over four things: (1) .. wait, let's use bullet points!</p>
<ul>
<li>The history, and the problem with Confidential Transactions</li>
<li>Pedersen commitments extended; an inner product proof</li>
<li>A very rough outline of how to use inner product proofs to make rangeproofs smaller</li>
<li>The scaling of this new version of CT; aggregation</li>
</ul>
<p>My role here is as an enthusiastic amateur; I mention this not because there's anything wrong with being an amateur per se, but more because really fully grokking the indepth details here is, at least so far, a bit beyond me, so this is somewhat of a skim, not a real exposition. Needless to say, all mistakes are my own.</p>
<p>Having said that, I have put together a very basic and rough proof of concept Python implementation <a href="https://github.com/AdamISZ/bulletproofspoc">here</a>; although it's only an implementation of the simplest case (nonaggregated, see below). This helped me to confirm my understanding of the algebraic execution of the algorithm; it may help you also if that's something that interests you.</p>
<h3>History</h3>
<p>Confidential Transactions was introduced by Greg Maxwell, and outlined for the general reader <a href="https://www.elementsproject.org/elements/confidentialtransactions/investigation.html">here</a>. If you haven't read that, you probably should. Greg gave a talk on it <a href="https://www.youtube.com/watch?v=LHPYNZ8i1cU">here</a>; I wrote an indepth investigation <a href="https://github.com/AdamISZ/ConfidentialTransactionsDoc">here</a> (aimed at those who don't know much about the ECC and other primitives, but builds up the full picture, so it's rather long). A shorter and more elegant overview was given <a href="http://cryptoservices.github.io/cryptography/2017/07/21/Sigs.html">here</a>, although it assumes a tadge more knowledge from the reader (and there are a couple of niggles in terms of how it's presented there, but I don't think they matter). As for implementations, there is that in the Elements project, and it was also folded into Monero's RingCT design as outlined <a href="https://lab.getmonero.org/pubs/MRL0005.pdf">here</a>.</p>
<p>A stupidly high level description would be: use Pedersen commitments to the amounts in the outputs of transactions instead of the plaintext amounts, then use the homomorphism of the Pedersen commitments (the fact that addition is preserved under the commitments) to check that the transaction is valid. Lastly, add in a "rangeproof" to ensure that the amounts (which perforce are integers modulo \(N\), where \(N\) is the size of the secp256k1 group ) don't overflow.</p>
<p>These "rangeproofs" are the big fly in the ointment of the construction  they, in the original implementation, are <strong>large</strong>. A typical output would have to be about 2500 bytes to encapsulate such a rangeproof, which is ten times larger than a typical Bitcoin transaction, in its entirety, today.</p>
<p>Incremental improvements were found (in particular, by Adam Back), and it's also true that the size was dependent on the granularity; the 2500 figure above was for 32 bit amounts, e.g. ~ 43btc with 1 satoshi granularity, but there wasn't a combination of (decent range of values to hide under) and (reasonable size of rangeproof) that seemed feasible for CT on Bitcoin, given how much of a premium there is, naturally, for space onchain.</p>
<p>Note that this is part of a broad theme; if, even with tweaks, privacyenabled transactions cost a lot of space, and therefore a lot of money, it degrades the anonymity set so extremely that it makes the privacy enhancement much, much weaker (albeit, nonzero still for sure!).</p>
<h3>Pedersen commitments and a direction to a new solution</h3>
<p>As a quick reminder, the basic structure of a Pedersen commitment is (using CT style notation):</p>
<p>\(C = xG + aH\)</p>
<p>where \(x\) is a "blinding" factor, and a is the amount under proof, while \(G\) is secp256k1's generator and \(H\) is an alternate generator (sometimes called "basepoint"), the ECdiscrete log for which is not known. I went into some considerable detail in describing all of this in my CT document linked above (section 2.12.2). As a recap, note that a commitment must have two properties: hiding, enabled by the blinding factor \(x\), which is random, and binding, which means you can't open the commitment to a value different to \(a\); this requires that the creator does not know the discrete log (=private key) of \(H\); if he did then he could do arithmetic on \(a\) and \(x\) and change them; if he doesn't, he can <em>only</em> open the commitment with exactly those two numbers and no other pair. If you think about it, that requires that <em>nobody</em> knows such a private key, for the system as a whole to work, this is addressed with the concept of "nothing up my sleeve" or <a href="https://en.wikipedia.org/wiki/Nothing_up_my_sleeve_number">NUMS</a>.</p>
<h4>Vector Pedersen commitments</h4>
<p>The first step forward is to consider multiple such NUMS generators. I have a memory back in 2015 of quipping on IRC about "vector spaces in ECC" (also called linear spaces), because after a while looking at these types of things you start to wonder ... since \(xG + aH\) is so useful (not just in CT), wouldn't \(xG + aH + bJ \ldots \) be useful too? The quip about vector spaces is observing that the structure \(x\textbf{e}_1 + y\textbf{e}_2,\ x,y \in \mathbb{R}\) defines a 2D space spanned by linearly independent basis vectors \(\textbf{e}_1,\textbf{e}_2\), and similarly for Ndimensional; but .. this is weird! The group of points on an elliptic curve is obviously only a 1D space (each point is defined by a single scalar \(\in \mathbb{Z}_n\)), and yet we can kind of "pretend" it's Ndimensional, in this context, <em>precisely</em> because we don't have a discrete log to convert between any two of these NUMS points.</p>
<p>Indeed, this kind of construction has already been used, afaik, in things like Confidential Assets (<a href="https://blockstream.com/bitcoin17final41.pdf">Blockstream impl., </a><a href="https://blog.chain.com/hiddeninplainsighttransactingprivatelyonablockchain835ab75c01cb">Chain impl.</a>). I had an intuition it should be possible/useful to leverage it in making another kind of range proof, although I had no idea how to do it.</p>
<p>So the vector pedersen commitment is the following natural extension of the previous:</p>
<p>\(C = xG + v_1H_1 + v_2H_2 + \ldots + v_nH_n\)</p>
<p>One blinding value is still sufficient here; the \(H_{i}\)s are just as many NUMS generators as we like. They can be created using a "coercehashtopoint" approach (again, see my CT doc for details; also see <a href="https://github.com/AdamISZ/bulletproofspoc/blob/master/utils.py#L59">here</a> in my bulletproofs code for one simpleenough way to create a whole set of them). This effectively creates a commitment to a vector of dimension N. And we can go further (indeed, bulletproofs does), and make a commitment to <em>multiple</em> vectors:</p>
<p>\(C = xU + v_1G_1 + v_2G_2 + \ldots + v_nG_n + w_1H_1 + w_2H_2 + \ldots + w_nH_n\)</p>
<p>It's not difficult to imagine that this much richer structure can allow more interesting algebraic constructions. But it has one very big drawback: opening the commitment means communicating the values committed to, which here would be \(2n + 1\) scalar values; in the case of a curve like secp256k1 that means communicating \((2n+1) \times 32\) bytes  a lot of data!</p>
<p>There's a subtletly here about commitments vs simply communicating data: if the commitment is part of a larger protocol, it may actually be acceptable to reveal the contents of the commitments (e.g. because the values inside are blinded with random values), but the commitment itself may still be important as part of that larger protocol (e.g. it is combined with other commitments). In this situation, transferring the contents (here it would be \(v_1, v_2, ..., w_n\) is an acceptable step to include, but the trouble is it may be a huge amount of data, for large values of \(n\).</p>
<p>More specifically, in Bulletproofs, we're going to focus on a way of proving that the dot product (or scalar product, or inner product) of the two vectors committed to has a certain value. Again, this can be achieved by simply revealing all the vector elements, but that's what we're going to try to avoid for compactness. Concretely, the commitment is structured as:</p>
<p>\(C = \mathbf{v} \cdot \mathbf{w} U + v_1G_1 + v_2G_2 + \ldots + v_nG_n + w_1H_1 + w_2H_2 + \ldots + w_nH_n\)</p>
<p>and a proof is created that the coefficient of \(U\) is indeed the dot product of the two vectors that were committed to against the preagreed generator set (\(U, \mathbf{G},\mathbf{H}\)) (bolding means vectors, even of EC points here).</p>
<h4>Making the proof smaller</h4>
<p>So how to avoid just sending all the data? <strong>The way that Bulletproofs constructs this proof involves halving the dimensions of these vectors by changing the basis repeatedly, until they boil down to single scalars, adding in an extra (2) commitments in each step. This reduces the data that needs to be transferred from \(\simeq n \times\ \) the size of the vectors to \(\simeq \textrm{log}_{2}(n) \times\ \)</strong>.</p>
<p>This central concept is a breakthrough from UCL cryptographers from last year (2016), Bootle et al. as per <a href="https://eprint.iacr.org/2016/263">this</a> paper. That paper explains the core idea in section 4.1, but in a form more complicated than that in Bulletproofs (section 3 and Protocol 1,2). Bootle also wrote a withoutmathematicsgist blog post <a href="https://www.benthamsgaze.org/2016/10/25/howtodozeroknowledgefromdiscretelogsinunder7kb/">here</a> which is quite helpful for the casual reader to get an idea.</p>
<p>How to understand this? Consider this problem: I want to prove to you I know 2 values, let's say. Suppose I already have a Pedersenstyle commitment of the form \(C = rG + aH_1 + bH_2\), then if I want to prove to you I know \(a\) and \(b\), I have to send you \(a\) <em>and</em> \(b\), right? Even though proving I know them is not <em>quite</em> the same thing as revealing them, I can't do better, can I? Revealing them means sending 2 x 32 bytes in our scenario. Wouldn't it be nice to just send 1x32 bytes? Well yeah but if I constructed \(rG + (a+b)H_1\) as the commitment instead, I haven't proven knowledge of \(a, b\) but only their sum.</p>
<p>But let's try to follow a wellknown pattern in cryptography (see <a href="https://en.wikipedia.org/wiki/Proof_of_knowledge#Sigma_protocols">sigma protocol</a>)  the verifier can send a random challenge to the prover. I send the commitment \(C\), you (verifier) send me a challenge random value \(x\). How can I use the \(x\) to reduce the amount of data I need to send to prove knowledge of the opening (\(a,b\), also known as the "witness") of the commitment?</p>
<p>A natural way is to send just \(ax + b\), but this is no good, because how does the verifier relate that to the originally sent \(C\)? It's no good if I can "open" a commitment to whatever I like! It must be binding. We need a construction, some \(f(a, b, x)\), that allows an algebraic reconstruction of \(C\) by the verifier, when combined with some function \(g(H_1,H_2)\). This is how the Bulletproofs paper ends up naturally with this arrangement:</p>
<p>\(a' = ax + bx^{1}\quad ,\ H' = x^{1}H_1 + xH_2\)</p>
<p>\(\therefore C' = (ax + bx^{1})(x^{1}H_1 + xH_2)\)</p>
<p>\(=aH_1 + bH_2 + x^2aH_2 + x^{2}bH_1 = C + x^2L + x^{2}R\)</p>
<p>(Note that these formulae ignore the blinding factor \(rG\), we will mention why in the next section).</p>
<p>What this means is that, proving knowledge of the opening of \(C\) means sending not \(a, b\) but instead \(a', L, R\) (note that the verifier has the challenge value \(x\) to so can recreate the final equation above).</p>
<p>But wait!  I hear you cry. We were trying to <em>reduce</em> the amount of data sent, but now we have to send two EC points \(L, R\) along with the now one scalar value \(a'\). True, this is not helpful; but the marvellous thing is that if you extend from single points \(a, b\) to entire vectors \(\textbf{a},\textbf{b}\) and use vector Pedersen commitments, you can reproduce the above process! The difference will be that, instead of "reducing" the transfer from \(a, b\) to \(ax + bx^{1}, L, R\), you'll be reducing the transfer from \(a_1, a_2, \ldots a_{n}\) to \(xa_1+x^{1}a_{\frac{n}{2}+1}, \ldots xa_{\frac{n}{2}} + x^{1}a_{n}, L, R\). Note how this is a new vector \(\textbf{a}'\) which is half the size of the original one.</p>
<p>At the end of that transformation, you've got a new vector, half the length of the first one, for which you're trying to prove knowledge of the entries; that's just a reduced version of the problem you started with, so you apply the protocol recursively. If you started with a power of 2, you can keep repeating the process until you have only 2 values you want to prove knowledge of, then one final step is required (as for the values \(a, b\) we started with). (see eqns 28 and 29 in the paper).</p>
<p>This means that you only have to <strong>send one pair of L and R points for each halving of the size of the vector, </strong>along with the final multiplied result.</p>
<h4>Using this trick for an inner product proof</h4>
<p>It's only a bit more complexity than the idea above, to include that the "blinding" factor in the vector pedersen commitment is actually the inner product of the two vectors, i.e. to prove that the commitment had structure:</p>
<p>\(C = \mathbf{v} \cdot \mathbf{w} U + v_1G_1 + v_2G_2 + \ldots + v_nG_n + w_1H_1 + w_2H_2 + \ldots + w_nH_n\)</p>
<p>The gist of it is to include the necessary crossterms in the formulae for the points \(L, R\) at each halving of the size of the vectors, such that at each step, it is always the case that \(c = \mathbf{v} \cdot \mathbf{w}\), and so in the final verification step (when vectors of length 2 are reduced to single values \(v, w\), the value is now \(c = v \times w\) as a direct multiplication. The details are in Protocol 1 in the paper, and you can see my simple implementation of proving and verifying, using recursive functions, <a href="https://github.com/AdamISZ/bulletproofspoc/blob/master/innerproduct.py#L64L103">here</a> and <a href="https://github.com/AdamISZ/bulletproofspoc/blob/master/innerproduct.py#L120L150">here</a>.</p>
<p>Protocol 2 is just a final tweak to allow not just proving that a commitment \(C\) has the above pattern, but that specifically the inner product is exactly a provided value \(c\).</p>
<h4>Quick aside  FiatShamir</h4>
<p>The above section explained crudely how you can use a "challenge value" \(x\), sent from verifier to prover, to condense the size of the final proof; in all such cases in cryptography we can replace such an interaction with the use of a cryptographic hash function applied to the previous steps of communication between prover and verifier, to make the proof noninteractive. See <a href="https://en.wikipedia.org/wiki/Fiat%E2%80%93Shamir_heuristic">here</a> for details. This is of course used and is briefly mentioned in section 4.4 of the paper.</p>
<h3>Rangeproofs from innerproductproofs</h3>
<p>This is actually the <em>more</em> complex part of the construction, compared to the above!</p>
<p>My description will be quite skimmy, partly because it's really a lot of algebra, and more importantly, because I don't understand it perfectly myself, yet.</p>
<p>We're going to assume that, as in CT currently, there is an existing Pedersen commitment of form \(C = xG + vH\), where \(v\) is the value of the transaction output in satoshis, and we want to prove that it's in range and doesn't overflow; typically we might want to prove that \(v \in 0 .. 2^{32}1 \), i.e. a 32 bit integer. In this case \(n=32\) in the below.</p>
<p>Bulletpoints time again!</p>
<ul>
<li>Start by making a vector called \(\textbf{a}_L\) which is a representation of \(v\) in bits, e.g. the value 3 might be represented by the vector [0,0,0,1,1] for proving it's in range 031 (this would be \(n=5\). Note that this means that \(\textbf{a}_L \cdot \textbf{2}^n = v\) (notation: \(\textbf{k}^n\) means the vector [\(1,k,k^2,\ldots ,k^{n1}\)] ).</li>
<li>Make a complementary vector \(\textbf{a}_R\) for which each component is the corresponding component of \(\textbf{a}_L\), minus 1. This means that \(\textbf{a}_L \cdot \textbf{a}_R = 0\).</li>
<li>Commit to these vectors with a commitment \(A\), and do the same for blinding vectors, creating \(S\). The verifier can respond to these commitments with two challenge values (FiatShamir comment applies of course).</li>
<li>The prover then builds two vectorvalued linear polynomials of a single scalar variable \(X\)  that is to say, the polynomials have coefficients that are vectors, so that when you plug in specific scalars for \(X\), you get vectors out  using the above variables. These polynomials are called \(l(X)\) and \(r(X)\). They are constructed so that their inner product \(t(X) = l(X) \cdot r(X)\), which is a quadratic polynomial in \(X\), has the following special property: its constant term \(t_0\) is a specific function of only \(v\) and the challenge values, <strong>if and only if </strong>the vectors \(\textbf{a}_L\), \(\textbf{a}_R\) were constructed honestly so that the previously mentioned properties hold.</li>
<li>The prover also constructs commitments to the other coefficients \(t_1, t_2\) of the polynomial and this data is sent to the verifier, who constructs more challenge values, and the data sent to the verifier is reduced by carefully constructing the commitment so that the verifier <span style="textdecoration: underline;">can be convinced of the fact that \(v \in 0..2^n 1\) by a proof that the value of t, for the challenged \(X\) value, is indeed the dot product: \(t(X) = l(X) \cdot r(X)\)</span>; and for this, the prover provides an inner product proof of the type described in the previous section, whose size is logarithmic in \(n\). Since it's not directly obvious that the constructed EC point in the paper (equation (62)) is indeed the required commitment, I've added some algebraic working  see<sup>1</sup> Footnote 1, to help if you're trying to figure it out.</li>
</ul>
<p>Overall the data sent to the verifier (you can see concretely in my code <a href="https://github.com/AdamISZ/bulletproofspoc/blob/master/rangeproof.py#L127L134">here</a>), is something close to:</p>
<p><span class="pls">\(33\times 4 + 32\times 3 + (32\times 2 + 33\times 2\times \textrm{log}_{2}(n))\)</span></p>
<p><span class="pls"> in bytes. The log part is coming from the \(L, R\) points, which as you may remember, must be sent for each halving of the sizes of the vectors whose inner product we're proving. Notice that this is <strong>dramatically</strong> smaller than a rangeproof of the former type (based on Borromean ring signatures over the digits of the amount); plugging in \(n=32\) gives you 622 bytes instead of 20002500. Of course the real gains occur at higher range sizes, since the growth is logarithmic.<br/></span></p>
<h3>Aggregation</h3>
<p>What neither my POC code, nor this brief writeup, have covered is the very important point: these rangeproofs can be <strong>aggregated</strong>, which can mean that a transaction with multiple outputs can create a proof of all of those multiple outputs being in range, with an additional size which is only logarithmic, effectively a constant extra term \(\simeq k \times \textrm{log}_{2}(m)\) where \(m\) is the number of outputs being proved. A reduction from 2500 to 650, say, is already big enough that CT may go from being impractical to practical; but aggregation could create scenarios where the number of bytes required <em>per output</em> in a transaction is really very small indeed; even 100 bytes or less, as we scale up to bigger and bigger numbers of outputs.</p>
<p>Note in particular that this <strong>dramatically economically incentivizes coinjoin</strong>; it makes it significantly cheaper to do your transaction along with someone else, at least if you compared with a CT model where rangeproofs were not aggregatable. The aggregation is of course interactive but as Greg notes in his overview on the mailing list:</p>
<blockquote>This scheme also has a straightforward and efficient method for multiparty computation, which means that the aggregates can be used in allpartyprivate coinjoins ...</blockquote>
<p>And the general positive point about CT + coinjoin still applies, which is that there is no need to match amounts; just coinjoin with anyone at all and you get the same effect. I think if you can also achieve the trick of making coinjoin noninteractive (see previous blog post on SNICKER), it might make coinjoin the default transaction type.</p>
<h3>Footnotes</h3>
<p>1. Demonstration that the constructed EC point in (62) of the Bulletproofs paper is the same as the commitment in (63) under honest prover conditions. Before we start, note that, following the convention in the paper (except here I use EC rather than exponent notation), a term like \(\textbf{aG}\) (note both scalar and point terms are bolded), actually refers to \(a_1G_1 + a_2G_2 + \ldots + a_nG_n\). Also recall as mentioned above that \(\textbf{k}^n\) (note exponent is <em>not</em> bolded), means the vector \([1, k, k^2, \ldots k^{n1}]\). We begin the construction in (62):</p>
<p>\(A = \left(a_{L1}G_1 + a_{L2}G_2 + \ldots + a_{Ln}G_n\right) + \left(a_{R1}H_1 + a_{R2}H_2 + \ldots + a_{Rn}H_n\right) + \alpha H \)</p>
<p>\(xS = \left(xs_{L1}G_1 + xs_{L2}G_2 + \ldots + xs_{Ln}G_n\right) + \left(xs_{R1}H_1 + xs_{R2}H_2 + \ldots + xs_{Rn}H_n\right) + x \rho H\)</p>
<p>\(\because \mu = \alpha + \rho X \therefore\)</p>
<p>\(A + xS z\textbf{G} = \mu H + \left(\left(a_{L1} + xs_{L1} z\right)G_1 + \ldots + \left(a_{Ln} + xs_{Ln} z\right)G_n\right)\)</p>
<p>\(\quad + \left(\left(a_{R1} + xs_{R1}\right)H_1 + \ldots + \left(a_{Rn} + xs_{Rn}\right)H_n\right)\)</p>
<p>We can now note that the term \(\left(\left(a_{L1} + xs_{L1} z\right)G_1 + \ldots + \left(a_{Ln} + xs_{Ln} z\right)G_n\right)\) in the above, is exactly the vector \(\textbf{l}(x)\)</p>
<p>In addition to the above, the point \(P\) in (62) has the additional term \(\left(z\textbf{y}^n + z^{2}\textbf{2}^n\right) \textbf{H}'\). The definition of \(\textbf{H}'\) is:</p>
<p>\(\textbf{H}' = [1H_1, y^{1}H_2, y^{2}H_3, \ldots , y^{(n1)}H_n]\)</p>
<p>So the additional term has form (note \(\circ\) is the Hadamard product as defined in the paper):</p>
<p>\(z\textbf{H} + \textbf{y}^{n} \circ \left(z^{2}\textbf{2}^n\right)\textbf{H}\)</p>
<p>So we now have:</p>
<p>\(A + xS z\textbf{G} + \left(z\textbf{y}^n + z^{2}\textbf{2}^n\right) \textbf{H}' = \mu H + \textbf{l}(x)\textbf{G} + \ldots\)</p>
<p>\(\quad \left(\textbf{a}_R + x\textbf{s}_R + z.\textbf{1}^n + \textbf{y}^{n} \circ \left(z^{2}\textbf{2}^n\right)\right)\textbf{H} \)</p>
<p>, the last term of which, according to the definition of \(\textbf{H}'\), can be rewritten as:</p>
<p>\(\left(\textbf{y}^n \circ \left(\textbf{a}_R + x\textbf{s}_R + z.\textbf{1}^n\right) + z^{2}\textbf{2}^n\right)\textbf{H}'\)</p>
<p>but this is exactly: \(\textbf{r}(x)\textbf{H}'\) . So finally we have that the RHS of (62) can be written as:</p>
<p>\(P = \mu H + \textbf{l}(x)\textbf{G} + \textbf{r}(x)\textbf{H}' \)</p>
<p>, which is the same form as (63). Final note: to convert this into the required the verifier must subtract \(\mu H\) and add back in \(t U\) to convert into the required form for verification of the inner product proof.</p>
<p></p>SNICKER20171026T11:16:04+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/snicker/<h2>SNICKER  Simple NonInteractive Coinjoin with Keys for Encryption Reused</h2>
<p></p>
<p>I'm going to do this backwards  start with the end goal user experience, and then work backwards to the technical design. This way, those not wanting to get lost in technical details can still get the gist.</p>
<h3><img alt="Me misusing a meme as a symbol and not adding any text." height="330" src="https://joinmarket.me/static/media/uploads/.thumbnails/evilplanbaby.jpg/evilplanbaby400x330.jpg" width="400"/></h3>
<p><em>Pictured above: me misusing a meme as a symbol and deliberately not adding any text to it.</em></p>
<h3><strong>Scenario</strong></h3>
<p><strong>Alisa</strong> lives in Moscow; she is a techsavvy Bitcoin user, uses Linux and the command line, and runs a fully verifying Bitcoin Core node. She doesn't have indexing enabled, but she (sometimes, or longrunning) runs a tool called <code>snickerscan</code> on the blocks received by her node. It scans recent Bitcoin blocks looking for transactions with a particular pattern, and returns to her in a file a list of candidate transactions. She pipes this list into another tool which uses her own Bitcoin wallet and constructs proposals: new transactions involving her own utxos and utxos from these newly found transactions, which she signs herself. Then, for each one, she makes up a secret random number and sends (the proposed transactions + the secrets), encrypted to a certain public key, in each case, so no one but the owner can read it, to a Tor hidden service which accepts such submissions. For now, her job is done and she gets on with her day.</p>
<p><strong>Bob</strong> lives in New York. He's a Bitcoin enthusiast who uses it a lot, and likes to test out new features, but has never written code and isn't techsavvy like that. A few hours after Alisa went to bed he opens one of his mobile wallets and a message pops up: <code>New coinjoin proposals found. Check?</code>. He heard about this, and heard that you can improve your privacy with this option, and even sometimes gain a few satoshis in the process. So he clicks <code>Yes</code>. In the background his mobile wallet downloads a file of some 510MB (more on this later!). Bob did this once before and was curious about the file; when he opened it he saw it was text with lots of unintelligible encrypted stuff like this:</p>
<p><code>QklFMQOVXvpqgjaJFm00QhuJ1iWsnYYV4yJLjE0LaXa8N8c34Hzg5CeQduV.....</code><br/><code>QklFMQI2JR50dOGEQdDdmeX0BwMH4c+yEW1v5/IyT900WBGdYRA/T5mqBMc.....</code></p>
<p>Now his mobile does some processing on this file; it takes a little while, some seconds perhaps, processing in the background. At the end it pops up a new message: <code>Coinjoin transaction found. Would you like to broadcast it?</code> and underneath it shows the transaction spending 0.2433 BTC out of his wallet and returning 0.2434 BTC in one of the outputs. It shows that the other inputs and outputs are not his, although one of them is also for 0.2434 BTC. Does he want to accept? Sure! Free money even if it's only cents. Even with no free money, he knows that coinjoin makes his privacy better. So he clicks <code>Yes</code> and it's broadcast. Done.</p>
<h3>The NIC in SNICKER</h3>
<p>Noninteractivity is a hugely desirable property in protocols; this is particularly the case where privacy is a priority. Firstly, it avoids the need to synchronize (<strong>Alisa</strong>, and her computer, had gone to sleep when <strong>Bob</strong> performed his step). Second, to avoid malicious interruption of an interactive protocol, it can help to identify the participants, but that is very damaging to the whole point of a protocol whose goal is privacy. Noninteractivity cuts this particular Gordian knot; one side can send the message anonymously and the other participant simply uses the data, but this has the limitation of the sender finding the receiver, which means some weak identification of the latter. Even better is if the request can be sent encrypted to the receiver, then it can be broadcast anywhere for the receiver to notice. That latter model is the most powerful, and is used here, but it does have practicality drawbacks as we'll discuss.</p>
<p>So, note that in the above scenario <strong>Alisa</strong> and <strong>Bob </strong>do not meet, do not synchronize, and need never meet or find out who each other are in future either. Their "meeting" is entirely abstracted out to one side publishing an encrypted message and the other side receiving <em>all</em> such encrypted messages and only reading the one(s) encrypted to his pubkey. The <em>all</em> part helps preserve Bob's privacy, if he finds a way to broadcast the final transaction with a reasonable anonymity defence (see e.g. <a href="https://github.com/gfanti/bips/blob/master/bipdandelion.mediawiki">Dandelion</a>; I'm of the opinion that that battle  making Bitcoin transaction broadcast anonymous  is something we <em>will</em> win, there is a massive asymmetry in favour of the privacy defender there).</p>
<h3>Quick background  how to do a Coinjoin</h3>
<p>Here's the obligatory <a href="https://bitcointalk.org/index.php?topic=279249.0">link</a> to the Coinjoin OP. You can skip this section if you know Coinjoin well.</p>
<p>Otherwise, I'll give you a quick intro here, one that naturally leads into the SNICKER concept:</p>
<p>Each input to a transaction requires (for the transaction to be valid) a signature by the owner of the private key (using singular deliberately, restricting consideration to p2pkh or segwit equivalent here) over a message which is ~ the transaction. Each of these signatures can be constructed separately, by separate parties if indeed the private key for each input are owned by separate parties. The "normal" coinjoining process thus involves the following steps (for now, not specifying <em>who</em> carries out each step):</p>
<ul>
<li>Gather all of the inputs  the utxos that will be spent</li>
<li>Gather all of the destination addresses to various parties, and the amounts to be paid</li>
<li>Distribute a "template" of the transaction to all parties (i.e. the transaction without any signatures)</li>
<li>In some order all of the parties sign the transaction; whomever has a transaction with all signatures complete, can broadcast it to the Bitcoin network</li>
</ul>
<p>There are different protocols one can choose to get all these steps done, ranging from simple to complex. A server can be the coordinating party; blinding can be used to prevent the server knowing inputoutput mapping. <a href="http://crypsys.mmci.unisaarland.de/projects/CoinShuffle/">Coinshuffle</a> can be used, creating a kind of onionrouting approach to prevent parties involved knowing the linkages (doesn't require a server to coordinate, but requires more complex interactivity). One of the parties in the join can be the "server", thus that party gains privacy that the others don't (Joinmarket). Etc.</p>
<p>The difficulties created by any interactivity are considerably ameliorated in a clientserver model (see e.g. the old blockchain.info <a href="https://en.bitcoin.it/wiki/Shared_coin">SharedCoin</a>(link outdated) model), the serious tradeoff is the server knowing too much, and/or a coordination/waiting problem (which may be considered tolerable; see both SharedCoin and <a href="https://github.com/darkwallet/darkwallet">DarkWallet</a>; with a sufficient liquidity pool the waiting may be acceptable).</p>
<p>There are a lot of details to discuss here, but there is always <em>some</em> interactivity (you can only sign once you know the full transaction, assuming no custom sighashing<sup>1</sup>), and a model with a server is basically always going to be more problematic, especially at scale.</p>
<p>So hence we try to construct a way of doing at least simple Coinjoins, in at least some scenarios, without any server requirement or coordination. Now I'll present the basic technical concept of how to do this in SNICKER, in 2 versions.</p>
<h3>First version  snicKER = Keys for Encryption Reused</h3>
<p>To make the Coinjoin noninteractive, we need it to be the case that Alisa can post a message for Bob, without explicitly requesting to create a private message channel with him. This requires encrypting a message that can then be broadcast (e.g. over a p2p network or on a bulletin board).</p>
<p><em>(In case it isn't clear that either encryption or a private message channel is required, consider that Alice must pass to Bob a secret which identifies Bob's output address (explained below), critically, and also her signature, which is on only her inputs; if these are seen in public, the inputoutput linkages are obvious to anyone watching, defeating the usual purpose of Coinjoin.)</em></p>
<h5>Encryption</h5>
<p>To achieve this we need a public key to encrypt a message to Bob. This is the same kind of idea as is used in tools like PGP/gpg  only the owner of the public key's private key can read the message.</p>
<p>In this "First version" we will assume something naughty on Bob's part: that he has <strong>reused an address</strong>! Thus, a public key will exist on the blockchain which we assume (not guaranteed but likely; nothing dangerous if he doesn't) he still holds the private key for.</p>
<p>Given this admittedly unfortunate assumption, we can use a simple and established encryption protocol such as <a href="https://en.wikipedia.org/wiki/Integrated_Encryption_Scheme">ECIES</a> to encrypt a message to the holder of that public key.</p>
<p>Alisa, upon finding such a pubkey, call it <code>P<sub>B</sub></code>, and noting the corresponding utxo <code>U<sub>B</sub></code>, will need to send, ECIES encrypted to <code>P<sub>B</sub></code>, several items (mostly wrapped up in a transaction) to Bob to give him enough material to construct a valid coinjoin without any interaction with herself:</p>
<ul>
<li>Her own utxos (just <code>U<sub>A</sub></code> for simplicity)</li>
<li>Her proposed destination address(s)</li>
<li>Her proposed amounts for output</li>
<li>Her proposed bitcoin transaction fee</li>
<li>The full proposed transaction template using <code>U<sub>A</sub></code> and <code>U<sub>B</sub></code> as inputs (the above 4 can be implied from this)</li>
<li>Her own signature on the transaction using the key for <code>U<sub>A</sub></code></li>
<li>Her proposed destination address <strong>for Bob</strong>.</li>
</ul>
<h4>Destination</h4>
<p>The last point in the above list is of course at first glance not possible, unless you made some ultra dubious assumptions about shared ownership, i.e. if Alisa somehow tried to deduce other addresses that Bob already owns (involving <em>more</em> address reuse). I don't dismiss this approach <em>completely</em> but it certainly looks like a bit of an ugly mess to build a system based on that. Instead, we can use a very well known construct in ECC; in English something like "you can tweak a counterparty's pubkey by adding a point that <em>you</em> know the private key for, but you still won't know the private key of the sum". Thus in this case, Alice, given Bob's existing pubkey <code>P<sub>B</sub></code>, which is the one she is using to encrypt the message, can construct a new pubkey:</p>
<pre><code>P<sub>B2</sub> = P<sub>B</sub> + k*G
</code></pre>
<p>for some 32 byte random value <code>k</code>.</p>
<p>Alice will include the value of <code>k</code> in the encrypted message, so Bob can verify that the newly proposed destination is under his control (again we'll just assume a standard p2pkh address based on <code>P<sub>B2</sub></code>, or a segwit equivalent).</p>
<p>Assuming Bob somehow finds this message and successfully ECIESdecrypts it using the private key of <code>P<sub>B</sub></code>, he now has everything he needs to (if he chooses), sign and broadcast the coinjoin transaction.</p>
<h4>A protocol for the most naive version, in broad strokes:</h4>
<ol>
<li>Alisa must have the ability to scan the blockchain to some extent; she must find scriptSigs or witnesses containing pubkeys which were later reused in new addresses/scriptPubKeys.</li>
<li>Alisa will use some kind of filtering mechanism to decide which are interesting. The most obvious two examples are: amounts under control in Bob's utxos matching her desired range, and perhaps age of utxos (so likely level of activity of user) or some watermarking not yet considered.</li>
<li>Having found a set of potential candidates, for each case <code>P<sub>B</sub>, U<sub>B</sub></code>: Construct a standard formatted message; here is a simple suggestion although in no way definitive:</li>
</ol>
<pre><code> 8(?) magic bytes and 2 version bytes for the message type
kvalue 32 bytes
Partially signed transaction in standard Bitcoin serialization
(optionally padding to some fixed length)
</code></pre>
<p>We defer discussing how in practice Bob will get access to the message later; but note that if he has done this, he already knows the value of <code>P_B</code> and will thus know also <code>U_B</code>. He ECIESdecrypts it, and recognizes it's for him through correct magic bytes (other messages encrypted to other pubkeys will come out random).</p>
<p>Then, this format has sufficient information for Bob to evaluate easily. First, he can verify that <code>U_B</code> is in the inputs. Then he can verify that for 1 of the 2 outputs (simple model) has a scriptPubKey corresponding to <code>P<sub>B2</sub> = P<sub>B</sub> + k*G</code>. He can then verify the output amounts fit his requirements. Finally he can verify the ECDSA signature provided on <code>U_A</code> (hence "partially signed transaction"). Given this he can, if he chooses, sign on <code>U<sub>B</sub></code> using <code>P<sub>B</sub></code> and broadcast. He must of course keep a permanent record of either <code>k</code> itself or, more likely, the private key <code>k + x</code> (assuming <code>P = x * G</code>).</p>
<h3>A proofofconcept</h3>
<p>Before going further into details, and discussing the second (probably superior but not as obviously workable) version of SNICKER, I want to mention that I very quickly put together some proof of concept code in <a href="https://github.com/AdamISZ/SNICKERPOC">this github repo</a>; it uses <a href="https://github.com/JoinmarketOrg/joinmarketclientserver">Joinmarketclientserver</a> as a dependency, implements ECIES in a compatible form to that used by <a href="https://electrum.org">Electrum</a>, and allows testing on regtest or testnet, admittedly with a bunch of manual steps, using the python script <code>snickertool.py</code>. The workflow for testing is in the README. To extend the testing to more wallets requires some way to do ECIES as well as some way to construct the destination addresses as per <code>P<sub>B2</sub> = P<sub>B</sub> + kG</code> above. I did note that, usefully, the partially signed transactions can be signed directly in Bitcoin Core using <code>signrawtransaction</code> and then <code>sendrawtransaction</code> for broadcast, but note that somehow you'll have to recover the destination address, as receiver, too. Note that there was no attempt at all to construct a scanning tool for any reusedkey transactions here, and I don't intend to do that (at least, in that codebase).</p>
<h2>Practical issues</h2>
<p>In this section will be a set of small subsections describing various issues that will have to be addressed to make this work.</p>
<h3>Wallet integration</h3>
<p>One reason this model is interesting is because it's much more plausible to integrate into an existing wallet than something like Joinmarket  which requires dealing with long term interactivity with other participants, communicating on a custom messaging channel, handling protocol negotiation failures etc. To do SNICKER as a receiver, a wallet needs the following elements:</p>
<ul>
<li>ECIES  this is really simple if you have the underlying secp256k1 and HMAC dependencies; see <a href="https://github.com/spesmilo/electrum/blob/master/lib/bitcoin.py#L774L817">here</a> and <a href="https://github.com/AdamISZ/SNICKERPOC/blob/master/ecies/ecies.py#L10L50">here</a>; note that the root construction in ECIES is ECDH.</li>
<li>The ability to calculate <strong>and store</strong> the newly derived keys of the form <code>P' = P + kG</code> where <code>k</code> is what is passed to you, and <code>P</code> is the pubkey of your existing key controlling the output to be spent. I would presume that you would have to treat <code>k+x</code>, where <code>P=xG</code>, as a newly imported private key. Note that we <em>cannot</em> use a deterministic scheme for this from <code>P</code>, since that would be calculatable by an external observer; it must be based on a secret generated by "Alisa".This could be a bit annoying for a wallet, although of course it's easy in a naive sense.</li>
<li>Ability to parse files containing encrypted coinjoin proposals in the format outlined above  this is trivial.</li>
<li>Ability to finish the signing of a partially signed transaction. Most wallets have this out of the box (Core does for example); there might be a problem for a wallet if it tacitly assumes complete ownership of all inputs.</li>
</ul>
<p>If a wallet only wanted to implement the receiver side (what we called "Bob" above), that's it.</p>
<h4>Compatibility/consensus between different wallets</h4>
<p>The only "consensus" part of the protocol is the format of the encrypted coinjoin proposals (and the ECIES algorithm used to encrypt them). We could deal with different transaction types being proposed (i.e. different templates, e.g. 3 outputs or 4, segwit or not), although obviously it'll be saner if there are a certain set of templates that everyone knows is acceptable to others.</p>
<h3>Notes on scanning for candidates</h3>
<p>There is no real need for each individual "Alisa" to scan, although she might wish to if she has a Bitcoin node with indexing enabled. This is a job that can be done by any public block explorer and anyone can retrieve the data, albeit there are privacy concerns just from you choosing to download this data. The data could be replicated on Tor hidden services for example for better privacy. So for now I'm assuming that scanning, itself, is not an issue.</p>
<p>A much bigger issue might be finding <strong>plausible</strong> candidates. Even in this version 1 model of looking only for reused keys, which are hopefully not a huge subset of the total utxo set, there are tons of potential candidates and, to start with, none of them at all are plausible. How to filter them?</p>
<ul>
<li>Filter on amount  if Alisa has X coins to join, she'll want to work with outputs < X.</li>
<li>Filter on age  this is more debatable, but very old utxos are less likely to be candidates for usage.</li>
<li>An "active" filter  this is more likely to be how things work. Are certain transactions intrinsically watermarked in a way that indicates that the "Bob" in question is actually interested in this function? One way this can happen is if we know that the transaction is from a certain type of wallet, which already has this feature enabled.</li>
</ul>
<h4>Bootstrapping</h4>
<p>If a set of users were using a particular wallet or service (preferably a <em>large</em> set), it might be possible to identify their transactions "Acme wallet transactions". Funnily enough, Joinmarket, because it uses a set and unusual coinjoin pattern, satisfies this property in a very obvious way; but there might be other cases too. See the notes in "second version", below, on how Joinmarket might work specifically in that case.</p>
<p>Better of course, is if we achieved that goal with a more userfriendly wallet with a much bigger userbase; I'd ask wallet developers to consider how this might be achieved.</p>
<p>Another aspect of bootstrapping is the Joinmarket concept  i.e. make a financial incentive to help bootstrap. If creators/proposers are sufficiently motivated they may offer a small financial incentive to "sweeten the pot", as was suggested in the scenario at the start of this post. This will help a lot if you want the userset to grow reasonably large.</p>
<h3>Scalability</h3>
<p>This is of course filed under "problems you really want to have", but it's nevertheless a very real problem, arguably the biggest one here.</p>
<p>Imagine 10,000 utxo candidates that are plausible and 1000 active proposers. Imagine they could all make proposals for a largeish subset of the total candidates, we could easily imagine 1,000,000 candidates at a particular time. Each encrypted record takes 500800 bytes of space, let's say. Just the data transfer starts to get huge  hundreds of megabytes? Perhaps this is not as bad as it looks, <em>if</em> the data is being received in small amounts over long periods.</p>
<p>And let's say we can find a way to get the data out to everybody  they still have to try to decrypt <strong>every</strong> proposal with <strong>every</strong> pubkey they have that is a valid candidate (in version 1, that's reused keys, let's say, or some subset of them). The computational requirement of that is huge, even if some cleverness could reduce it (decrypt only one AES block; use high performance C code e.g. based on libsecp256k1). Again, perhaps if this is happening slowly, streamed over time, or in chunks at regular integrals, it's not as bad. Still.</p>
<p>It's true that these problems don't arise at small scale, but then the real value of this would be if it scaled up to large anonymity sets.</p>
<p>Even if this is addressed, there is another problem arising out of the anonymous submission  any repository of proposals could be filled with junk, to waste everyone's time. Apart from a <a href="https://en.wikipedia.org/wiki/Hashcash">hashcash</a>like solution (not too implausible but may impose too much cost on the proposer), I'm not sure how one could address that while keeping submission anonymity.</p>
<p>At least we have the nice concept that this kind of protocol can improve privacy on Bitcoin's blockchain without blowing up bandwidth and computation for the Bitcoin network itself  it's "offband", unlike things like <a href="https://www.elementsproject.org/elements/confidentialtransactions/investigation.html">Confidential Transactions</a> (although, of course, the effect of that is much more powerful). I think ideas that take semantics and computation off chain are particularly interesting.</p>
<h3>Conflicting proposals</h3>
<p>This is not really a problem: if Alisa proposes a coinjoin to Bob1 and Bob2, and Bob1 accepts, then when Bob2 checks, he will find one of the inputs for his proposed coinjoin is already spent, so it's not valid. Especially in cases where there is a financial incentive, it just incentives Bobs to be more proactive, or just be out of luck.</p>
<h3>Transaction structure and 2 party joins</h3>
<p>We have thus far talked only about 2 party coinjoins, which <em>ceteris paribus</em> are an inferior privacy model compared to any larger number (consider that in a 2 party coinjoin, the <em>other</em> party necessarily knows which output is yours). The SNICKER model is not easily extendable to N parties, although it's not impossible. But DarkWallet used 2 of 2 joins, and it's still in my opinion valuable. Costs are kept lower, and over time these joins heavily damage blockchain analysis. A larger number of joins, and larger anonymity set could greatly outweigh the negatives<em>.</em></p>
<p>Structure: the model used in the aforementioned <a href="https://github.com/AdamISZ/SNICKERPOC">POC</a>, although stupid simple, is still viable: 2 inputs, one from each party (easily extendable to 1+N), 3 outputs, with the receiver getting back exactly one output of ~ the same size as the one he started with. The proposer then has 1 output of exactly that size (so 2 equal outputs) and one change. Just as in Joinmarket, the concept is that fungibility is gained specifically in the equal outputs (the "coinjoin outputs"); the change output is of course trivially linked back to its originating input(s).</p>
<p>But there's no need for us to be limited to just one transaction structure; we could imagine many, perhaps some templates that various wallets could choose to support; and it'll always be up to the receiver to decide if he likes the structure or not. Even the stupid X>X, Y>Y "coinjoin" I mused about in my Milan presentation <a href="https://youtu.be/IKSSWUBqMCM?t=47m21s">here</a>(warning:youtube) might be fun to do (for some reason!). What a particularly good or "best" structure is, I'll leave open for others to discuss.</p>
<h3>Second version  snicKER = Keys Encrypted to R</h3>
<p>We've been discussing all kinds of weird and whacky "NonInteractive Coinjoin" models on IRC for years; and perhaps there will still be other variants. But arubi was mentioning to me yesterday that he was looking for a way to achieve this goal <em>without</em> the nasty requirement of reused keys, and between us we figured out that it is a fairly trivial extension, <em>if</em> you can find a way to get confidence that a particular existing utxo is coowned with an input (or any input). That's because if you have an input, you have not only a pubkey, but also a <strong>signature</strong> (both will either be stored in the scriptSig, or in the case of segwit, in the witness section of the transaction). An <a href="https://en.wikipedia.org/wiki/Elliptic_Curve_Digital_Signature_Algorithm">ECDSA</a> signature is published on the blockchain as a pair: <code>(r, s)</code>, where <code>r</code> is the xcoordinate of a point <code>R</code> on the secp256k1 curve. Now, any elliptic curve point can be treated as a pubkey, assuming someone knows the private key for it; in the case of ECDSA, we call the private key for <code>R</code>, <code>k</code>, that is: <code>R = kG</code>. <code>k</code> is called the nonce (="number used once"), and is usually today calculated using the algorithm <a href="https://tools.ietf.org/html/rfc6979">RFC6979</a>, which determines its value deterministically from the private key you're signing with, and the message. But what matters here is, the signer either already knows <code>k</code>, or can calculate it trivially from the signing key and the transaction. This provides us with exactly the same scenario as in the first version; Bob knows the private key of <code>R</code>, so Alisa can send a proposal encrypted to that public key, and can derive a new address for Bob's destination using the same formula:</p>
<pre>P<sub>B2</sub> = R + k'G</pre>
<p>Here I used <code>k'</code> to disambiguate from the signature nonce <code>k</code>, but it's exactly the same as before. As before, Bob, in order to spend the output from the coinjoin, will need to store the new private key <code>k+k'</code>. For a wallet it's a bit more work because you'll have to keep a record of past transaction <code>k</code> values, or perhaps keep the transactions and retrieve <code>k</code> as and when. Apart from that, the whole protocol is identical.</p>
<h4>Finding candidates in the second version</h4>
<p>In version 2, we no longer need Bob to do something dubious (reusing addresses). But now the proposer (Alisa) has a different and arguably harder problem than before; she has to find transactions where she has some reasonable presumption that a specific output and a specific input are coowned. You could argue that this is good, because now Alisa is proposing coinjoins where linkages <em>are</em> known, so she's improving privacy exactly where it's needed :) (only half true, but amusing). In a typical Bitcoin transaction there are two outputs  one to destination, one change; if you can unambiguously identify the change, even with say 90% likelihood not 100%, you could make proposals on this basis. This vastly expands the set of <em>possible </em>candidates, if not necessarily plausible ones (see above on bootstrapping).</p>
<p>Additionally paradoxical is the fact that Joinmarket transactions <em>do</em> have that property! The change outputs are unambiguously linkable to their corresponding inputs through subsetsum analysis, see e.g. <a href="https://github.com/AdamISZ/JMPrivacyAnalysis/blob/master/tumbler_privacy.md#jmsudokucoinjoinsudokuforjmtxs">here</a>.</p>
<p>Thus, Adlai Chandrasekhar's <a href="http://adlai.uncommonlisp.org:5000/">cjhunt</a> tool (appears down as of writing), <a href="https://github.com/adlai/cjhunt">code</a>, identifies all verylikelytobe Joinmarket transactions through blockchain scanning, and its output could be used to generate candidates (the proposed joins could be with those change outputs, using the `R` values from one of the identifiedascoowned inputs). See also <a href="https://citp.github.io/BlockSci/chain/blockchain.html">BlockSci</a>. Then if Joinmarket had both proposer and receiver side code integrated, it would create a scenario where these type of coinjoins would most likely be quite plausible to achieve.</p>
<h3>Conclusion</h3>
<p>I think this idea might well be viable. It's simple enough that there aren't likely crypto vulnerabilities. The short version of the pros and cons:</p>
<h4>Pros</h4>
<ul>
<li>No interactivity (the point), has many positive consequences, and high anonymity standard</li>
<li>Relative ease of wallet integration (esp. compared to e.g. Joinmarket), consensus requirement between them is limited.</li>
<li>Potentially huge anonymity set (different for version 1 vs version 2, but both very large)</li>
</ul>
<h4>Cons</h4>
<ul>
<li>For now only 2 parties and probably stuck there; limited coinjoin model (although many transaction patterns possible).</li>
<li>Finding plausible candidates is hard, needs a bootstrap</li>
<li>Sybil attack on the encrypted messages; how to avoid the "junk mail" problem</li>
</ul>
<p>Lastly, it should be fine with Schnorr (to investigate: aggregation in this model), in version 1 and version 2 forms.</p>
<h3>Footnotes</h3>
<p>1. Sighashing  attempting a noninteractive coinjoin with some interesting use of <code>SIGHASH_SINGLE</code> and <code>SIGHASH_ANYONECANPAY</code> seems at least plausible (see <a href="https://en.bitcoin.it/wiki/OP_CHECKSIG#Procedure_for_Hashtype_SIGHASH_SINGLE">here</a>), although it's not exactly heartening that no one ever uses <code>SIGHASH_SINGLE</code> (and its rules are arcane and restrictive), not to even speak of watermarking. Hopefully the idea expressed here is better.</p>Flipping the scriptless script on Schnorr20171011T09:41:24+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/flippingthescriptlessscriptonschnorr/<h2>Outline</h2>
<p>It's by now very well known in the community of Bitcoin enthusiasts that the <a href="https://en.wikipedia.org/wiki/Schnorr_signature">Schnorr signature</a> may have great significance; and "everyone knows" that its significance is that it will enable signatures to be aggregated, which could be <strong>*great*</strong> for scalability, and nice for privacy too. This has been elucidated quite nicely in a Bitcoin Core <a href="https://bitcoincore.org/en/2017/03/23/schnorrsignatureaggregation/">blog post</a>.</p>
<p>This is very true.</p>
<p>There are more fundamental reasons to like Schnorr too; it can be shown with a simple proof that Schnorr signatures are secure if the elliptic curve crypto that prevents someone stealing your coins (basically the "Elliptic Curve Discrete Logarithm Problem" or ECDLP for short) is secure, and assuming the hash function you're using is secure (see <a href="https://blog.cryptographyengineering.com/2011/09/29/whatisrandomoraclemodelandwhy3/">this deep dive into the random oracle model</a> if you're interested in such things). ECDSA doesn't have the same level of mathematical surety.</p>
<p>Perhaps most importantly of all Schnorr signatures are <strong>linear</strong> in the keys you're using (while ECDSA is not).</p>
<p>Which brings me to my lame puntitle : another way that Schnorr signatures may matter is to do with, in a sense, the <strong>opposite</strong> of Schnorr aggregation  Schnorr subtraction. The rest of this very long blog post is intended to lead you through the steps to showing how clever use of signature subtraction can lead to <span style="textdecoration: underline;">one</span> very excellent outcome (there are others!)  a private Coinswap that's simpler and better than the private Coinswap outlined in my <a href="https://joinmarket.me/blog/blog/coinswaps">previous blog post</a>.</p>
<p>The ideas being laid out in the rest of this post are an attempt to concretize work that, as far as I know, is primarily that of Andrew Poelstra, who has coined the term "<strong>scriptless scripts</strong>" to describe a whole set of applications, usually but not exclusively leveraging the linearity of Schnorr signatures to achieve goals that otherwise are not possible without a system like Bitcoin's <a href="https://en.bitcoin.it/wiki/Script">Script</a>. This was partly motivated by Mimblewimble (another separate, huge topic), but it certainly isn't limited to that. The broad overview of these ideas can be found in these <a href="https://download.wpsoftware.net/bitcoin/wizardry/mwslides/201705milanmeetup/slides.pdf">slides</a> from Poelstra's Milan presentation last May.</p>
<p>So what follows is a series of constructions, starting with Schnorr itself, that will (hopefully) achieve a goal: an onchain atomic coinswap where the swap of a secret occurs, on chain, inside the signatures  but the secret remains entirely invisible to outside observers; only the two parties can see it.</p>
<p>If you and I agree between ourselves that the number to subtract is 7, you can publish "100" on the blockchain and nobody except me will know that our secret is "93". Something similar (but more powerful) is happening here; remember signatures are actually just numbers; the reason it's "more powerful" is that we can enforce the revealing of the secret by the other party if the signature is valid, and coins successfully spent.</p>
<p></p>
<p>Before we therefore dive into how it works, I wanted to mention why this idea struck me as so important; after talking to Andrew and seeing the slides and talk referenced above, I <a href="https://twitter.com/waxwing__/status/862724170802761728">tweeted</a> about it:</p>
<p><strong>If we can take the <em>semantics</em> of transactions offchain in this kind of way, it will more and more improve what Bitcoin (or any other blockchain) can do  we can transact securely without exposing our contracts to the world, and we can reduce blockchain bloat by using secrets embedded in data that is already present. The long term vision would be to allow the blockchain itself to be a *very* lean contract enforcement mechanism, with all the "rich statefulness" .. clientside ;)<span style="textdecoration: underline;"><br/></span></strong></p>
<p></p>
<h4>Preliminaries: the Schnorr signature itself</h4>
<p><em>(Notation: We'll use <code></code> for concatenation and capitals for elliptic curve points and lower case letters for scalars.)</em></p>
<p>If you want to understand the construction of a Schnorr signature well, I can recommend Oleg Andreev's compact and clear <a href="http://blog.oleganza.com/post/162861219668/eli5howdigitalsignaturesactuallywork">description</a> ; also nice is Section 1 in the Maxwell/Poelstra Borromean Ring Signatures <a href="https://github.com/Blockstream/borromean_paper">paper</a>, although there are of course tons of other descriptions out there. We'll write it in basic form as:</p>
<pre><code>s = r + e * x
e = H(PRm)
</code></pre>
<p>Note: we can hash, as "challenge" a la <a href="https://en.wikipedia.org/wiki/Proof_of_knowledge#Sigma_protocols">sigma protocol</a>, just <code>Rm</code> in some cases, and more complex things than just <code>PRm</code>, too; this is just the most fundamental case, fixing the signature to a specific pubkey; the nonce point <code>R</code> is always required).</p>
<p>For clarity, in the above, <code>x</code> is the private key, <code>m</code> is the message, <code>r</code> is the "nonce" and <code>s</code> is the signature. The signature is published as either <code>(s, R)</code> or <code>(s, e)</code>, the former will be used here if necessary.</p>
<p>Apologies if people are more used to <code>s = r  ex</code>, for some reason it's always <code>+</code> to me!</p>
<p></p>
<p>Note the linearity, in handwavy terms we can say:</p>
<pre>s_1 = r_1 + e * x_1<br/>s_2 = r_2 + e * x_2<br/>e = H(P_1 + P2  R_1 + R_2  m)<br/>=><br/>s_1 + s_2 is a valid signature for public key (P_1 + P_2) on m.</pre>
<p>But this is <strong>NOT </strong>a useable construction asis: we'll discuss how aggregation of signatures is achieved properly later, briefly.</p>
<p></p>
<h4>Construction of an "adaptor" signature</h4>
<p>This is the particular aspect of Poelstra's "scriptless script" concept that gets us started leveraging the Schnorr signature's linearity to do fun things. In words, an "adaptor signature" is a not a full, valid signature on a message with your key, but functions as a kind of "promise" that a signature you agree to publish will reveal a secret, or equivalently, allows creation of a valid signature on your key for anyone possessing that secret.</p>
<p>Since this is the core idea, it's worth taking a step back here to see how the idea arises: you want to do a similar trick to what's already been done in atomic swaps: to enforce the atomicity of (spending a coin: revealing a secret); but without Script, you can't just appeal to something like <code>OP_HASH160</code>; if you're stuck in ECC land, all you have is scalar multiplication of elliptic curve points; but luckily that function operates similar to a hash function in being oneway; so you simply share an elliptic curve point (in this case it will be <code>T</code>), and the secret will be its corresponding private key. The beatiful thing is, it <em>is</em> possible to achieve that goal directly in the ECC Schnorr signing operation.</p>
<p>Here's how Alice would give such an adaptor signature to Bob:</p>
<p>Alice (<code>P = xG</code>), constructs for Bob:</p>
<ul>
<li>Calculate <code>T = tG</code>, <code>R = rG</code></li>
<li>Calculate <code>s = r + t + H(P  R+T  m) * x</code></li>
<li>Publish (to Bob, others): <code>(s', R, T)</code> with <code>s' = s  t</code> (so <code>s'</code> should be "adaptor signature"; this notation is retained for the rest of the document).</li>
</ul>
<p>Bob can verify the adaptor sig <code>s'</code> for <code>T,m</code>:</p>
<pre><code>s' * G ?= R + H(P  R+T  m) * P
</code></pre>
<p>This is not a valid sig: hashed nonce point is <code>R+T</code> not <code>R</code>;</p>
<p>Bob cannot retrieve a valid sig : to recover <code>s'+t</code> requires ECDLP solving.</p>
<p>After validation of adaptor sig by Bob, though, he knows:</p>
<p>Receipt of <code>t</code> <=> receipt of valid sig <code>s = s' + t</code></p>
<h4>Deniability:</h4>
<p>This is a way of concretizing the concept that all of this will be indistinguishable to an observer of the blockchain, that is to say, an observer only of the final fully valid signatures:</p>
<p>Given any <code>(s, R)</code> on chain, create <code>(t, T)</code>, and assert that the adaptor signature was: <code>s' = s  t</code>, with <code>R' = R  T</code>, so adaptor verify eqn was: <code>s'G = R' + H(P  R'+T  m)P</code></p>
<h4></h4>
<h4>Moving to the 2of2 case, with Schnorr</h4>
<p>For the remainder, we're considering the matter of signing off transactions from outpoints jointly owned (2 of 2) by Alice and Bob.</p>
<p>Start by assuming Alice has keypair <code>(x_A, P_A)</code>, and Bob <code>(x_B, P_B)</code>. Each chooses a random nonce point <code>r_A</code>, <code>r_B</code> and exchanges the curve points with each other (<code>P_A, R_A, P_B, R_B</code>) to create a scriptPubKey/destination address.</p>
<h4>2of2 Schnorr without adaptor sig</h4>
<p>To avoid relatedkey attacks (if you don't know what that means see e.g. the "Cancelation" section in <a href="https://diyhpl.us/wiki/transcripts/scalingbitcoin/milan/schnorrsignatures/">https://diyhpl.us/wiki/transcripts/scalingbitcoin/milan/schnorrsignatures/</a>), the "hash challenge" is made more complex here, as was noted in the first section on Schnorr signatures. The two parties Alice and Bob, starting with pubkeys <code>P_A</code>, <code>P_B</code>, construct for themselves a "joint key" thusly:</p>
<pre><code>P_A' = H(H(P_AP_B)  P_A) * P_A ,
P_B' = H(H(P_AP_B)  P_B) * P_B ,
joint_key = P_A' + P_B'
</code></pre>
<p>Note that Alice possesses the private key for <code>P_A'</code> (it's <code>H(H(P_AP_B)  P_A) * x_A</code>, we call it <code>x_A'</code> for brevity), and likewise does Bob. From now on, we'll call this "joint_key" <code>J(A, B)</code> to save space.</p>
<p>Common hash challenge:</p>
<pre><code>H(J(A, B)  R_A + R_B  m) = e
s_agg = = r_A + r_B + e(x_A' + x_B')
> s_agg * G = R_A + R_B + e * J(A, B)
</code></pre>
<p>Alice's sig: <code>s_A = r_A + e * x_A'</code>, Bob's sig: <code>s_B = r_B + e * x_B'</code> and of course: <code>s_agg = s_A + s_B</code>.</p>
<p>There is, as I understand it, more to say on this topic, see e.g.<a href="http://diyhpl.us/wiki/transcripts/bitcoincoredevtech/20170906signatureaggregation/">here</a>, but it's outside my zone of knowledge, and is somewhat orthogonal to the topic here.</p>
<h4>2of2 with adaptor sig</h4>
<p>Now suppose Bob chooses <code>t</code> s.t. <code>T = t * G</code>, and Bob is going to provide an adaptor signature for his half of the 2of2.</p>
<p>Then:</p>
<ol>
<li>Alice, Bob share <code>P_A, P_B, R_A, R_B</code> as above; Bob gives <code>T</code> to Alice</li>
<li>Alice and Bob therefore agree on <code>e = H(J(A, B)  R_A + R_B + T  m)</code> (note difference, <code>T</code>)</li>
<li>Bob provides adaptor <code>s' = r_B + e * x_B'</code> (as in previous section, not a valid signature, but verifiable)</li>
<li>Alice verifies: <code>s' * G ?= R_B + e * P_B'</code></li>
<li>If OK, Alice sends to Bob her sig: <code>s_A = r_A + e * x_A'</code></li>
<li>Bob completes, atomically releasing <code>t</code>: first, construct <code>s_B = r_B + t + e * x_B'</code>, then combine: <code>s_agg = s_A + s_B</code> and broadcast, then Alice sees <code>s_agg</code></li>
<li>Alice subtracts: <code>s_agg  s_A  s' = (r_B + t + e * x_B')  (r_B + e * x_B') = t</code></li>
</ol>
<p>Thus the desired property is achieved: <code>t</code> is revealed by a validating "completion" of the adaptor signature.</p>
<p><strong>Note</strong>, however that this has no timing control, Bob can jam the protocol indefinitely at step 6, forcing Alice to wait (assuming that what we're signing here is a transaction out of a sharedcontrol outpoint); this is addressed in the fleshed out protocol in the next section, though.</p>
<p>For the remainder, we'll call the above 7 steps the 22AS protocol, so <code>22AS(Bob,t, Alice)</code> for Bob, secret <code>t</code>, and Alice. Bob is listed first because he holds <code>t</code>.</p>
<p>Since this is the most important part of the construction, we'll summarize it with a schematic diagram:</p>
<p><img alt="22AS protocol" height="816" src="https://joinmarket.me/static/media/uploads/.thumbnails/22AS.jpg/22AS1056x816.jpg" style="display: block; marginleft: auto; marginright: auto;" width="1056"/></p>
<p>So this <code>22AS</code> was a protocol to swap a coin for a secret, to do atomic swaps we need to extend it slightly: have two transactions atomic via the same secret <code>t</code>.</p>
<h3>The Atomic Swap construct, using 2of2 schnorr + adaptor signatures</h3>
<p>This is now <em>fairly</em> straightforward, inheriting the main design from the existing "atomic swap" protocol.</p>
<p>A. Alice and Bob agree on a pair of scriptPubkeys which are based on 2 of 2 pubkeys using Schnorr, let's name them using <code>D</code> for destination address (<code>A</code> is taken by Alice): <code>D_1</code> being 22 on (<code>P_A1</code>, <code>P_B1</code>) and <code>D_2</code> being 22 on (<code>P_A2</code>, <code>P_B2</code>). Note that these pubkeys, and therefore destination addresses, are not dependent in any way on "adaptor" feature (which is a property only of nonces/sigs, not keys).</p>
<p>B. Alice prepares a transaction TX1 paying 1 coin into <code>D_1</code>, shares txid_1, and requires backout transaction signature from Bob. Backout transaction pays from txid_1 to Alice's destination but has locktime <code>L1</code>.</p>
<p>C. Bob does the (nearly) exact mirror image of the above: prepares TX2 paying 1 coin into <code>D_2</code>, shares txid_2, requires backout transaction signature from Alice. Backout transaction pays from txid_2 to Bob's destination with locktime <code>L2</code> which is <em>significantly later</em> than <code>L1</code>.</p>
<p>D. Then Alice and Bob broadcast TX1 and TX2 respectively and both sides wait until both confirmed. If one party fails to broadcast, the other uses their backout to refund.</p>
<p>E. If both txs confirmed (N blocks), Alice and Bob follow steps 14 of <code>22AS(Bob, t, Alice)</code> (described in previous section) for some <code>t</code>, for both the scriptPubkeys <code>D_1</code> and <code>D_2</code>, in parallel, but with the same secret <code>t</code> in each case (a fact which Alice verifies by ensuring use of same <code>T</code> in both cases). For the first (<code>D_1</code>) case, they are signing a transaction spending 1 coin to Bob. For the second, <code>D_2</code>, they are signing a transaction spending 1 coin to Alice. Note that at the end of these steps Alice will possess a verified adaptor sig <code>s'</code> for <em>both</em> of the spendouts from <code>D_1, D_2</code>.</p>
<p>E(a). Any communication or verification failure in those 14 steps (x2), both sides must fall back to timelocked refunds.</p>
<p>F. The parties then complete (steps 57) the first <code>22AS(Bob, t, Alice)</code> for the first transaction TX1, spending to <code>D_1</code> to give Bob 1 coin. Alice receives <code>t</code> as per step 7.</p>
<p>F(a). As was mentioned in the previous section, Bob can jam the above protocol at step 6: if he does, Alice can extract her coins from her timelocked refund from <code>D_1</code> in the period between <code>L1</code> and <code>L2</code>. The fact that <code>L2</code> is (significantly) later is what prevents Bob from backing out his own spend into <code>D_2</code> <em>and</em> claiming Alice's coins from <code>D_1</code> using the signature provided in step 5. (Note this time asymmetry is common to all atomic swap variants).</p>
<p>G. (Optionally Bob may transmit <code>t</code> directly over the private channel, else Alice has to read it from the blockchain (as per above <code>22AS</code> protocol) when Bob publishes his spend out of <code>D_1</code>).</p>
<p>H. Alice can now complete the equivalent of steps 57 without Bob's involvement for the second parallel run for <code>D_2</code>: she has <code>t</code>, and adds it to the already provided <code>s'</code> adaptor sig for the transaction paying her 1 coin from <code>D_2</code> as per first 4 steps. This <code>s' + t</code> is guaranteed to be a valid <code>s_B</code>, so she adds it to her own <code>s_A</code> to get a valid <code>s_agg</code> for this spend to her of 1 coin, and broadcasts.</p>
<p></p>
<h2>Summing up</h2>
<h4>Privacy implications</h4>
<p>In absence of backouts being published (i.e. in cooperative case), these scriptPubkeys will be the same as any other Schnorr type ones (N of N multisig will not be distinguishable from 1 of 1). The signatures will not reveal anything about the shared secret <code>t</code>, or the protocol carried out, so the 2 transaction pairs (payin to <code>D_1,D_2</code>, pay out from same) will not be tied together in that regard.</p>
<p>This construction, then, will (at least attempt to) gain the anonymity set of all Schnorr sig based transactions. The nice thing about Schnorr's aggregation win is, even perhaps more than segwit, the economic incentive to use it will be strong due to the size compaction, so this anonymity set should be big (although this is all a bit pie in the sky for now; we're a way off from it being concrete).</p>
<p>The issue of amount correlation, however, has <strong>not</strong> been in any way addressed by this, of course. It's a sidebar, but one interesting idea about amount correlation breaking was brought up by Chris Belcher <a href="https://github.com/AdamISZ/CoinSwapCS/issues/47">here</a> ; this may be a fruitful avenue whatever the flavour of Coinswap we're discussing.</p>
<h4>Comparison with other swaps</h4>
<p>Since we've now, in this blog post and the previous, seen 3 distinct ways to do an atomic coin swap, the reader is forgiven for being confused. This table summarizes the 3 different cases:</p>
<table border="1" height="147" style="height: 145px;" width="1034"><caption></caption>
<thead>
<tr style="backgroundcolor: #66d9ff;">
<td><strong>Type</strong></td>
<td><strong>Privacy onchain</strong></td>
<td><strong>Separate "backout/refund" transactions for noncooperation</strong></td>
<td><strong>Requires segwit</strong></td>
<td><strong>Requires Schnorr</strong></td>
<td><strong>Number of transactions in cooperative case</strong></td>
<td><strong>Number of transactions in noncooperative case</strong></td>
<td><strong>Space on chain</strong></td>
</tr>
</thead>
<tbody>
<tr>
<td>Atomic swap</td>
<td>None; trivially linkable</td>
<td>None; backout is directly in script</td>
<td>No</td>
<td>No</td>
<td>2 + 2</td>
<td>2 + 2</td>
<td>Medium</td>
</tr>
<tr>
<td>CoinSwap</td>
<td>Anonymity set: 2 of 2 transactions (+2 of 3 depending on setup)</td>
<td>Presigned backouts using H(X) and CLTV, break privacy if used</td>
<td>Yes</td>
<td>No</td>
<td>2 + 2</td>
<td>3 + 3</td>
<td>Largeish</td>
</tr>
<tr>
<td>Scriptless script</td>
<td>Anonymity set: all Schnorr transactions</td>
<td>Presigned backouts uslng locktime; semibreak privacy (other txs may use locktime)</td>
<td>Yes</td>
<td>Yes</td>
<td>2 + 2</td>
<td>2 + 2</td>
<td>Small</td>
</tr>
</tbody>
</table>
<p></p>
<p>The reason that there are "3 + 3" transactions in the noncooperative case for CoinSwap is, in that case, both sides pay into a 2of2, then in noncooperation, they must both spend into the custom "HTLC" (IF hash, pub, ELSE CLTV, pub), and then redeem *out* of it.</p>
<p>A fundamental difference for the latter 2 cases, compared with the first, is they must pay into sharedownership 2 of 2 outputs in the payin transaction; this is to allow backout transactions to be arranged (a twoparty multitransaction contract requires this; see e.g. Lightning for the same thing). The first, bare atomic swap is a single transaction contract, with the contract condtions embedded entirely in that one transaction(for each side)'s scriptPubKey.</p>
<p>Finally, size onchain of the transactions is boiled down to handwaving, because it's a bit of a complex analysis; the first type always uses a large redeem script but one signature on the payout, whether cooperative or noncooperative; the second uses 2 or 3 signatures (assuming something about how we attack the anonymity set problem) but no big redeem script in cooperative case, while takes up a *lot* of room in the noncooperative case, the third is always compact (even noncooperative backouts take no extra room). Schnorrsigscriptlessscripts are the big winner on space.</p>
<p></p>
<h4>Extending to multihop; Lightning, Mimblewimble</h4>
<p>The first time I think this was discussed was in the mailing list post <a href="https://lists.launchpad.net/mimblewimble/msg00086.html%20">here</a>, which discusses how conceivably one could achieve the same setup as HTLC for Mimblewimble lightning, using this scriptlessscriptatomicswap. Doubtless these ideas are a long way from being fleshed out, and I certainly haven't kept up with what's going on there :)</p>
<h4>Other applications of the scriptless script concept</h4>
<p>As a reminder, this document was just about fleshing out how the atomic swap gets done in a Schnorrsignaturescriptlessscript world; the <a href="https://download.wpsoftware.net/bitcoin/wizardry/mwslides/201705milanmeetup/slides.pdf">slides</a> give several other ideas that are related. Multisignature via aggregation is of course part of it, and is already included even in the above protocol (for 2 of 2 as a subset of N of N); earlier ideas like paytocontracthash and signtocontracthash already exist, and don't require Schnorr, but share a conceptual basis; same for ZKCP, etc.</p>
<h4>Cross chain swap</h4>
<p>I admit to not sharing <em>quite</em> the same breathless excitement about crosschain swaps as some people, but it is no doubt very interesting, if somewhat more challenging (not least because of different "clocks" (block arrivals) affecting any locktime analysis and confirmation depth). Poelstra has however also made the very intriguing point that it is <strong>not</strong> actually required for the two blockchains to be operating on the same elliptic curve group for the construction to work.</p>
<p></p>Coinswaps20170629T08:05:52+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/coinswaps/<h2>Preamble</h2>
<p>Firstly, I am a rubbish salesman, as well as a rubbish blog poster (first post in ~ 10 months!), mentioning this upfront  this is not a pitch for anything and indeed it may be an extremely long post about things that are utterly irrelevant (in the sense that most of what's discussed here is old, and has <em>partly</em> been superseded). But if you are curious about the tricks that can be played with Bitcoin's scripting, especially re: privacy enhancement, but don't have time to master it (join the club), this might prove of interest.</p>
<p>And humour aside, I think there is certainly still a case to be made that these protocols may have a place that they find use.</p>
<p></p>
<h2>Summary</h2>
<p>This post is an attempt to lead the curious but <strong>nonexpert</strong> reader to the point of understanding what "trustless coinswap" protocols in Bitcoin (and its variants) exist, how they work, how they differ, and what role they might serve. I've put in some links to relevant background so you can chase things up in more detail if you're interested. As the post progresses, things slowly get more technical so feel free to drop out once you've got the gist from the first 4ish sections if you get bored.</p>
<p>Bear in mind this is not a complete science; it's still evolving. Also, there is no guarantee of absence of mistakes, I will endeavour to update if they're found.</p>
<p>I want to also note that this post will <em>not</em> discuss Lightning or Tumblebit (see <a href="https://joinmarket.me/blog/blog/tumblebitforthetumblecurious/">previous post</a> from last year)  both of which are much more complex protocols using something <em>similar </em>to what you see here as a building block. See the section "Aside  HTLC". This is about the low level, simpler idea of "atomically swapping coins".</p>
<h3>Atomicity</h3>
<p>The etymologists among you will already know that <strong>atom </strong>comes from Greek and means 'a' not/un, 'tom' cut, in other words, not cuttable or divisible. The idea of transactions being <em>atomic</em> in that sense  not being splittable into parts, "all or nothing", is well known in the world of databases. And Bitcoin, being a fancy (or just plain weird) kind of database, the same concept applies. When Alice pays Bob and Charlie in <em>one</em> transaction, they all know that either <em>both </em>Bob and Charlie get paid, or neither. A transaction confirms or it doesn't, it doesn't halfconfirm. Quite apart from Alice, Bob or Charlie, the ledger itself needs to make sure money doesn't get created out of thin air, so atomicity is as fundamental as it gets (ask <a href="https://en.wikipedia.org/wiki/The_DAO_(organization)">these guys</a>).</p>
<p></p>
<p>When you have atomicity it makes life easier for customers, merchants, software developers ... everyone really. When you don't, life is harder. A good exampe of nonatomic transactions is buying a second hand mobile phone from a web store. You send them money, then they send you the phone. Maybe the phone didn't arrive, oops. Wasn't atomic, so trust and other messy symptoms get involved.</p>
<p></p>
<h3>The notevenaswap swap; self"mixing"</h3>
<p>Coinbase might ban you for gambling. Send your coins to different addresses ... but hmm, they're all your addresses. Can Coinbase tell you've done that? Here's a better question: why the heck are you using Coinbase? :)</p>
<p>What you really want is that the coins you give to Coinbase have <em>no</em> connection, on the blockchain, to those you received from inveterategamblers.com. You <span style="textdecoration: underline;">can't</span> get that exact effect by sending coins to yourself. Although, you can get something close using Coinjoin, but it takes time and money to do it (see: Joinmarket).</p>
<h3>The nonatomic swap, aka the mixer</h3>
<p>You can give your coins to an online (usually hidden service on Tor) mixer service. They give you back <em>different coins</em>, that is to say, the coins you get back have a different history, not a direct connection to the ones you put in. Coinbase feel better now? Maybe, maybe not.</p>
<p></p>
<p>But the in and out phase are not <em>atomic</em>, as discussed. Coins may go in, coins may not go out. Even Bill O'Reilly can explain that.</p>
<p></p>
<h3>Atomic swap without privacy features</h3>
<p>I've led up to this rather slowly to make sure no one is lost by just jumping in. But this is the heart of the matter and it's the one thing you should make sure to take away, even if you get bored by all the technical stuff after it.</p>
<p></p>
<blockquote>
<div>We can use a <strong>cryptographic commitment</strong> scheme to create atomicity that binds <strong>two, independent</strong> Bitcoin transactions, as long as we assume Bitcoin basically works (i.e. that transactions cannot be censored, or reversed)</div>
</blockquote>
<p>The way to do this for Bitcoin is to use a hash function (say, Bitcoin's HASH160 (which is RIPEMD160 on top of SHA256). Suppose Alice and Bob want to create this atomicity binding two transactions, one created by each. The way to do it is to have one of them  say Alice  make up a random number, \(x\) and hash it, say \(H(x)\). We'll make \(x=\)<code>deadbeef</code>, although the real one needs to be properly random. Then make the scripts on the outputs of the transactions look like:</p>
<pre>HASH160 9ecd593e655360e068c597e157651bef911dc597 EQUALVERIFY alicepubkey CHECKSIG</pre>
<p>(Alice pays 1BTC into a P2SH address for the above script)</p>
<pre>HASH160 9ecd593e655360e068c597e157651bef911dc597 EQUALVERIFY bobpubkey CHECKSIG</pre>
<p>(Bob pays 1BTC into a P2SH address for the above script)</p>
<p><span style="textdecoration: underline;"><em>This is deliberately oversimplified for now  don't do it!</em></span>. In English, I hope you can see that this <em>basically </em>means "you can claim these coins ONLY if you can give me an \(x\) such that its hash equals 9ecd..97 <strong>and</strong> you make a valid signature for the pubkey" (the "make valid signature" part is just the same as a normal transaction; you have to sign it to authorise it; the hash value part is an <strong>extra</strong> requirement you have to provide).</p>
<p></p>
<p>So in that environment:</p>
<blockquote>
<p>What makes the two transactions atomic is that if Alice ever spends her output (claims her coins), she has to <strong>PUBLISH</strong> \(x\) on the blockchain, which perforce gives Bob the information he needs to spend <em>his </em>coins!</p>
<p></p>
</blockquote>
<p>Who said Bitcoin was not for publishing data? ;) <em></em>So, that's the basic idea. From now on, we go into details:</p>
<p>First, and hence the title of this section, notice that <strong>actually publishing transactions spending from scripts like that unambiguously connects the two transactions on the blockchain</strong> for anyone scanning/reading it. So a protocol that simply publishes them is not achieving any privacy gain in a meaningful sense; but then, as we shall see, this protocol might not be mainly about privacy. Let's flesh out the protocol.</p>
<p></p>
<p>It's generally accepted that the origin of concept of "atomic swaps" is in this <a href="https://bitcointalk.org/index.php?topic=193281.0">TierNolan post</a>. Unfortunately the thread is a bit of a difficult read, and various alternatives are proposed  the main idea is there but it isn't clear, plus the analysis was done before CLTV/CSV existed. Fortunately Nicolas Dorier <a href="https://github.com/AdamISZ/CoinSwapCS/issues/25#issuecomment311281096">recently</a> boiled it down and showed what I <em>believe </em>to be the simplest way to do it on github for me/us. I'll reproduce here:</p>
<p><span style="textdecoration: underline;"> </span></p>
<p>Alice chooses random number \(x\). Makes hash \(H(x)\). Gives Bob \(H(x)\).</p>
<ul>
<li>Alice pays 1 coin to this script (wrapped into a P2SH address, of course), <strong>redeemscript1</strong>:</li>
</ul>
<pre>IF HASH160 H(x) EQV bobpubkey CHECKSIG ELSE 1000 CHECKLOCKTIMEVERIFY DROP alicepubkey CHECKSIG ENDIF</pre>
<p>(In English this, and the next, script pay to "Either one pubkey if you can provide the hash preimage (\(x\)) or the other pubkey after the timeout block". "EQV" is short for <code>OP_EQUALVERIFY</code> . For the timeout part, see <a href="https://github.com/bitcoin/bips/blob/master/bip0065.mediawiki">bip65</a> for details on <code>CHECKLOCKTIMEVERIFY</code>. And note there is also <a href="https://github.com/bitcoin/bips/blob/master/bip0112.mediawiki">this</a>: <code>CHECKSEQUENCEVERIFY.</code>)</p>
<ul>
<li>Alice waits for that transaction to confirm, then tells Bob about it and waits for his side.</li>
</ul>
<ul>
<li>And Bob pays 1 coin to <strong>redeemscript2</strong>:</li>
</ul>
<pre>IF HASH160 H(x) EQV alicepubkey CHECKSIG ELSE 950 CHECKLOCKTIMEVERIFY DROP bobpubkey CHECKSIG ENDIF</pre>
<ul>
<li>and waits for it to confirm, and tells Alice about it. Now, Alice can spend out of Bob's output (the second one above), because she already knows \(x\).</li>
<li>Bob is monitoring that output of course, so when it gets spent he looks at the scriptSig and will see:</li>
</ul>
<pre>alicesignature x 1 redeemscript1</pre>
<ul>
<li>(the '1' here can be used to choose the "IF" branch instead of the "ELSE"; there's a better way of doing this, but ignore that for now) ... from which he extracts x, then simply does the same thing to redeem 1 coin from the output Alice created:</li>
</ul>
<pre>bobsignature x 1 redeemscript2</pre>
<p>If Alice or Bob don't make those payments and instead back out after blocks 1000/950 respectively, then the other side can likewise back out after that time, getting back their coins.</p>
<p><span style="textdecoration: underline;"> </span></p>
<p>An essential part of the logic here is that <strong>Bob can spend his timelock backout before Alice</strong> (see 950 vs 1000). And if Alice doesn't show the secret to him he <em>should</em> spend using the timelock before Alice's timelock (i.e. between 950 and 1000) to be safe. It's a bit confusing at first, but you can see this as the counterpoint to the fact that Alice has the secret before Bob; so she initiates any spend via the secret branches; so if the first block Bob could spend out of his timeout were later than Alice, Alice could steal both the coins (one via secret, one via timeout) in that intervening period (he'd have no chance winning a race by reading the secret off the blockchain; she'd have sent both out <strong>simultaneously</strong>).</p>
<p></p>
<p>Now, notice that since the two transaction paths are not connected, there's no reason that they even have to be on the same blockchain. Both blockchains have to support verification of the hash function and prevention of spending before a certain time/block. To keep the assessment simple, just note that it can be done on LTC and BTC (as Nicolas mentioned in his summary).</p>
<p></p>
<p>And indeed this is the obvious application of this protocol: there isn't much point creating disconnected histories on one blockchain (or is there?) if you both use the same secret value \(x\) since that unambiguously links them (\(x\) will perforce be a highentropy random number and thus completely unique to that tx pair). But being able to swap, say, my BTC for your LTC <strong>without either of us trusting each other at all </strong>is quite a significant thing to be able to do (in fact, I still remember that the first time I was shown CoinSwap (see next section), apart from being quite bewildered, the only thing I <em>did</em> immediately understand was that this could be done across two different blockchains!).</p>
<p></p>
<h3>Aside  the HTLC</h3>
<p>Before going on to discuss the privacy enhanced variant, let's observe that the special scriptPubkey, or "redeeming condition", that we brought in in the last section  the "IF HASH THEN THIS PUBKEY SIGNS ELSE AFTER BLOCK N, THIS OTHER PUBKEY SIGNS" to put it crudely, has become known, at least informally, as the "Hashed Timelock Contract" or HTLC; at least, according to <a href="https://en.bitcoin.it/wiki/Hashed_Timelock_Contracts">this link</a>, although I get the impression the <em>exact</em> definition may change with who you talk to, and the term seems to have originated with the development of the Lightning protocol (please correct that if I'm wrong!). What's probably more important is this  Lightning is not about <em>swapping</em> coins but rather (partly) about <em>routing</em> them through payment channels with multiple hops. The idea of secretreveal (hash preimage) is there extended  instead of just 2 parties having both transactions dependent on a single secret \(x\), you can thus have a whole chain of them. This can be thought of as <strong>spreading the atomicity not just over two transactions, but several. </strong>I won't say more here ... this is a huge topic; perhaps this idea might whet your appetite.</p>
<h3>Coinswap  with privacy</h3>
<p>Back to swapping coins.</p>
<p>Greg Maxwell in 2013 proposed <a href="https://bitcointalk.org/index.php?topic=321228.0">Coinswap</a>, which is different to the "Atomic Cross Chain Swap" as described above, but based on the same core idea of publicationofsecretenforcesatomicity. As you'll see from the "Motivation", the main difference is that it proposes to actually create extra privacy/fungibility, basically by <strong>not publishing the hashedsecret scripts similar to the above, but keeping them as a backout, and prefering to publish a nonsecretcontaining transaction.</strong> It was published before the advent of <code>CHECKLOCKTIMEVERIFY</code> or <code>CHECKSEQUENCEVERIFY</code> , and so used only the existing nLockTime feature in Bitcoin at the time. But also it had a flaw in it, at least as originally laid out (which I've explained <a href="https://github.com/AdamISZ/CoinSwapCS/blob/master/docs/coinswap_tweak.md">here</a>, for those interested). In attempting to modify it using CLTV, I ended up with a structure similar to the previous section, but with another layer.</p>
<p></p>
<p>Here is a diagram (taken from <a href="https://github.com/AdamISZ/CoinSwapCS/blob/master/docs/coinswap_new.pdf">here</a>); the participants are "Alice" and "Carol" (no Bob):</p>
<p><img alt="coinswapnewbasic" height="549" src="https://joinmarket.me/static/media/uploads/.thumbnails/coinswapforblog1.jpg/coinswapforblog1974x549.jpg" width="974"/></p>
<p></p>
<p>The left hand side is the coins funded by Alice; the right hand side is those funded by Carol. The transactions at the bottom (TX2, TX3) play a similar role to the ones explained in detail in the previous section, and indeed, they have exactly the same redemption scripts (both "IF hash160 hashval EQV onepubkey checksig ELSE blocktime CLTV (drop) otherpubkey checksig ENDIF"), and just as for the "Atomic swap" design, the L0 must be after the L1, where L0 and L1 are the earliest block height each of Alice and Carol respectively can use that timelock branch to recover coins. The reason L0 must be after L1 is exactly as before.</p>
<p>But what's different here is that, if all goes well, TX2 and TX3 <span style="textdecoration: underline;">will never be broadcast on the blockchain</span>. Instead, as long as both sides know they <strong>can</strong> broadcast them  which means both sides have to sign them in advance (they <strong>both</strong> sign both notice  because the inputs are from 2 of 2 addresses under their joint control), they can choose instead to broadcast TX4 and TX5 as replacements; they still get the same amount of coins, but these transactions have no common \(x\) value, and no custom redeem scripts, so <strong>in principle</strong> (we'll come to this) they can look like any other p2sh spending transaction on the blockchain, i.e. a snooper may not be able to see either (a) that they are transactions of "coinswap" type or (b) that the two branches are in any way connected.</p>
<p>It sounds nice having this extra layer, but it makes analysis much more tricky. The first tricky thing is that you <strong>must completely sign the backout transactions TX2 and TX3 before you even broadcast the funding transactions TX0,1</strong>. This is because, if you just broadcast TX0 immediately, you have locked your funds into a 2 of 2 address with your counterparty, and they are under no obligation to do anything at that point (or they could die or disappear etc.). This is clearly not riskless. Hence TX2/TX3 signing, completely, first. And whilst this is a very smart solution, it brings up a major issue (see next section).</p>
<p>There's a lot of detail that can't be covered here. If you happen to be interested please do some review of the material in the documentation folder in my <a href="https://github.com/AdamISZ/CoinSwapCS/blob/master/docs">CoinSwapCS repo</a>. For example, there is an attempt to iterate all backout cases for each side. However, see the last two sections of this post first.</p>
<p>Here's an example of two transactions representing two sides backing out via TX2 and TX3 (an example of a bunch of tests I've been doing on testnet): <a href="https://www.blocktrail.com/tBTC/tx/ec432451317b03064cfcf3ce5c26a2e71f30f8ae83a6f6f63f7b1ba0c4739dc8?txinIdx=0">https://www.blocktrail.com/tBTC/tx/ec432451317b03064cfcf3ce5c26a2e71f30f8ae83a6f6f63f7b1ba0c4739dc8?txinIdx=0</a> and <a href="https://www.blocktrail.com/tBTC/tx/c14c14fa37a57ba3ddf716fb8d960ac3ccdb9bc049b744002efed2c771454664">https://www.blocktrail.com/tBTC/tx/c14c14fa37a57ba3ddf716fb8d960ac3ccdb9bc049b744002efed2c771454664</a> ; these are links directly to the final transactions that <em>redeem</em> the outputs of TX2 and TX3; you can trace back and see TX0,1 and TX2,3 and notice that the amounts are a bit more complicated than you expect, due to the fees arrangements outlined in the docs. But in any case you can see the redeem scripts basically follow the logic above  you can see the secret value \(x\) (it's 14 bytes) on both sides, and you also see a little extra feature around OP_DEPTH.</p>
<h3>Advantages</h3>
<p>This document is trying to very briefly survey the ideas, because I think they <em>might</em> be important, rather than advertise any one as particularly good, but at least somewhere here I should record why CoinSwap as described above might be interesting: basically it's about anonymity set, in other words, the size of the crowd you can hide in. <strong>IF</strong> (big if!) a protocol like that can hide in <strong>all</strong> of the p2sh users by only spending coins from 2 of 2 or 2 of 3, it provides something very much not provided by things like coinjoin.</p>
<p>The other advantage is transaction size. The level of fungibility achieved per byte on the blockchain should be high, although it's difficult to quantify.</p>
<p>But, these advantages do not obtain if (a) you are forced into backout transactions and/or (b) there is trivial correlation on amount, or time such that an observer can easily relink the two paths.</p>
<p>Finally, an independent advantage to the others  certainly if comparing with Coinjoin  is that this, like the previous nonprivate version, is entirely possible to do between different blockchains, albeit the complexity disadvantage of making it private carries over there too.</p>
<h3>Malleability rears its head</h3>
<p>For the CoinSwap (privacy) design mentioned above, a nasty issue arises in the preparation of the backouts you don't intend to spend (if the other side cooperates). To create them you must give the <strong>txid</strong> of your initial spending transaction (in the diagrams, TX0 and TX1) to your counterparty; that's part of the transaction template; and ask them to sign. If after you do this, you broadcast say TX0, but then your counterparty is malicious (or a miner is) and TX0 is malleated to a different txid, your backout transaction has become useless; it's got a signature on an input that doesn't, and can't exist.</p>
<p>What kind of malleability are we talking about here? A miner can push a highS signature onto the chain; these are nonstandard but valid (post BIP66 activation). But see <a href="https://github.com/bitcoin/bips/blob/master/bip0062.mediawiki">BIP62</a> for some examples of other ways malleability can be achieved via the exact construction of the "scriptSig" (technical name for the data provided by the spender to redeem the coins; for normal addresses it's just (ecdsa signature, ecdsa pubkey), but it can have other things in it). One example is adding opcodes like <code>OP_NOP8</code> which are there for future upgrades and literally just do nothing to the script. But their presence as a single extra byte <em>completely</em> changes the txid of the transaction (remember that's a key property of hash functions). And again BIP62 shows us there are <em>several</em> possible sources of this kind of malleability.</p>
<p>Things *could* still go fine if such a malleation occurs by a 3rd party, if your counterparty cooperates; but they could also hold you up and e.g. demand half your coins (worst case scenario, but .. ouch! that's not what we're here for...).</p>
<p></p>
<p>BIP141 <a href="https://github.com/bitcoin/bips/blob/master/bip0141.mediawiki">Segwit</a> solves this problem, of course, because its specific purpose is to remove txid malleability. In the absence of segwit, there <a href="https://bitcointalk.org/index.php?topic=303088.0">are ideas</a> about how one can get round this partially by means of arranging it so that no one *knows* which TX being broadcast is actually your TX0, so it would take huge resources to deliberately attack you. But these <a href="https://github.com/AdamISZ/CoinSwapCS/issues/25#issuecomment311793482">ideas</a> are a bit fragile, if still somewhat interesting.</p>
<p></p>
<p>Also note that <strong>no such malleability issue can arise</strong> with the much simpler "atomic swap" idea that was discussed first, because there is no separate backout path that has to be signed in advance to be safe.</p>
<p></p>
<h3>Another privacy tweak appears impossible for now</h3>
<p>One thing that would be cool is to take the first of the two above versions (the "Atomic Swap") but somehow tweak it so that the secret values \(x\) appearing in the two broadcast spends were different. But they'd have to still be inextricably linked in that: when Alice spends with \(x\), Bob knows he can spend with \(y\), even though to an outside observer \(x\) and \(y\) were completely random and unrelated. This would <em>at least</em> extend the anonymity set to everyone doing such swaps, instead of creating an immediate link. Suppose that our hash function \(H\) obeyed \(H(a+b) = H(a) + H(b)\); then Alice could privately share \(b\) with Bob, along with \(H(a)\), and Bob's transaction could output to \(H(a)+H(b)\) without him knowing \(a\) in advance (as previously he did not know \(x\)).Then when Alice paid to her output, exposing \(a\), he can add \(a+b\) to find the necessary secret to redeem his output. But on the blockchain the secrets shown are \(a\) on one side, and \(a+b\) on the other, which have no publically evident connection (remember \(b\) was transmitted privately).</p>
<p>Unfortunately hash functions don't behave like that. Elliptic curve points do; \(aG + bG = (a+b)G\) and so the reader can easily see that the above mechanism would work fine if we replaced \(H\) with the operation of elliptic curve point scalar multiplication, i.e. the function that converts a private key to a public one. But this is no good for Bitcoin Script, because EC operations like that are not available within it. So no dice for now, although it could be the basis for interesting ideas in future. By the way I was informed (by Andrew Poelstra) that this exact line of thought has been gone through somewhere on the Lightning mailing list, although I haven't chased up the link.</p>
<h2>Conclusions</h2>
<ul>
<li>These are pretty old concepts (20122014); but they're not easy to implement or use, which probably explains their lack of use</li>
<li>The core concept (see "HTLC") continues to be very interesting, powerful and makes up a part of the more modern/complex/potentially powerful ideas like Lightning/Tumblebit</li>
<li>There is still a case to be made for possible usability of both the Atomic Cross Chain Swap (a very simple protocol that allows trustless exchange across blockchains) and the CoinSwap (more complex but potentially adding fungibility) either on same chain Bitcoin or across chains.</li>
<li>An advantage of Coinswap over Coinjoin is a potentially bigger anonymity set (a lot more could be said)</li>
<li>The privacy focused Coinswap has a very serious malleability problem without Segwit (there's a complex and somewhat fragile, partial solution)</li>
</ul>
<p>Congratulations (commiserations?) if you got this far. If you feel so inclined have chat with me on freenode (user waxwing) about all this weird stuff (or any bitcoin type place, really). I would reenable comments but I'd need captchas first, sorry (bots!).</p>TumbleBit for the tumblecurious20160930T13:35:50+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/tumblebitforthetumblecurious/<p>New ideas about achieving fungibility in Bitcoin or similar systems are coming at a furious rate; it's pretty difficult to keep up. One idea that was published recently I found to be particularly interesting  <a href="https://eprint.iacr.org/2016/575">TumbleBit</a>. The linked paper has the full description, both theoretical and practical, and even links to working <a href="https://github.com/BUSEC/TumbleBit">code</a> (that's a bit of a rarity unfortunately). If you're capable of reading the paper in full, you won't really need this blogpost, but I know most aren't, yet some are still very curious, so here we are.</p>
<p>Maybe even better than the working code is the POC tumble transactions, linked in the github <a href="https://github.com/BUSEC/TumbleBit">README</a>, that show how the model has been used to delink 800 inbound payments from 800 outbound (equal sized) payments in a single tumble run. This nicely illustrates how the blockchainlevel separation achieved gives rise to a great scalabilityofmixing win.</p>
<p>Apologies in advance for the length! The intention is to get into the guts of the protocol, although missing a few details in that respect. If the post achieves anything, it's mainly a matter of translating the mathematics into English; hopefully it can convince you that what TumbleBit achieves, while seeming very implausible at first glance, does actually work, practically. If it inspires you, I'd like to strongly encorage reading the original paper; I'd also strongly welcome any corrections!</p>
<h3>Goal of the system</h3>
<h4>1. The classic tumbler model</h4>
<p>Existing bitcoin "mixers", "washers", "tumblers" have a fully centralized model: not only centralized in that clients/customers do all negotiation with a central server, but most importantly <em>trust</em> that central server with their funds, and their information. The negatives of this model are so obvious that we needn't dwell on them here; the positives are however worth reflecting on  centralized services solve a coordination problem, but in the context of Bitcoin mixing a more profound issue is addressed  mixing <em>within one transaction </em>(see Coinjoin, including all variants) does not <em>break </em>linkages so much as <em>fuse</em> them; a confusion effect. A mixer, albeit it's very hard to do this well, can completely break linkages between money sent in and money sent out. Of course like any anonymity supporting system, it is limited by the anonymity set (if you are the only user of mixer server A you're not getting great anonymity!), but in principle it has that much more powerful effect of link "severage". To be fair, CoinSwap also has this property, but I haven't investigated this enough to comment yet (the paper does make a few high level comparative comments).</p>
<p>TumbleBit proposes to remove the main negative feature of the 'standard' mixer model  the trust (both types  trust in holding funds, trust in knowing linkages), while still keeping the breaking of linkages onchain. In its first proposed model, which it calls the "classic tumbler model", <strong>a set of users can join in a tumbling session, send in coins, wait, and then receive completely disconnected coins later</strong>  without worrying that the tumbler's going to run away with the money or know whose coins are whose.</p>
<p></p>
<h4>2. The unlinkable payments hub model</h4>
<p>I won't be investigating this in this document; refer to the original paper for more details.</p>
<p></p>
<h3>The crypto, dumbed down</h3>
<p>Here's the "magic" version. Even if you don't read the main paper, take a look at Fig. 1 in reference to this, it may help it to make more sense; although it'll take the rest of this blog post to properly explain it:</p>
<ul>
<li>Alice puts 1 coin into escrow with T</li>
<li>T puts 1 coin into escrow with Bob</li>
<li>Bob gets an encrypted signature on a transaction to pay<br/> him (Bob) 1 coin from the escrow with T. He can't yet broadcast/spend it; Bitcoin itself doesn't handle encrypted signatures!</li>
<li>Bob sends the encrypted (and blinded) decryption key for the signature to Alice.</li>
<li>Alice does a "fair exchange" to get the decryption key<br/>for the encrypted signature from T, in exchange for paying T 1 coin, from her escrow.</li>
<li>Alice can effectively pay Bob by handing over the decryption key (it's still blinded).</li>
<li>If Bob gets the blinded decryption key, he can unblind it, then use it to decrypt the encrypted signature for his payout.</li>
<li>Then Bob broadcasts his payout to the blockchain, receiving 1 coin. This transaction is completely separate from Alice's payin.</li>
</ul>
<p>Now let's go through the pieces that need explaining:</p>
<h4>Escrow transactions</h4>
<p>The escrow transactions have a highlevel logic like this:</p>
<pre>IF: time is past refund time: allow spends back to escrow creator<br/>ELSE IF: correct hash preimages provided: allow spend to the receiver.</pre>
<p>or:</p>
<pre><code>IF: time is past refund time: allow spends back to escrow creator<br/>ELSE IF: transaction is signed by 2 of 2 (creator, receiver)</code></pre>
<p>Basically in this kind of escrow, you commit the funds by actually spending into an address that is redeemed by a Bitcoin script like (one of) the above; you can be safe in the knowledge that if your counterparty disappears, or you choose not to pay, you can just grab the coins back any time after the expiry of the "refund time" (implemented using Bitcoin's <a href="https://github.com/bitcoin/bips/blob/master/bip0065.mediawiki">OP_CHECKLOCKTIME_VERIFY</a> in TumbleBit). On the other hand, the "other guy" (which could be one of the T or Bob in the highlevel description above) can take the money if certain conditions are fulfilled. You can see that an escrow transaction in itself is not enough for "trustless" exchange, but it takes a key role in setting that up.</p>
<h5>Technical detail  Bitcoin scripts:</h5>
<p>For reference, here is the Bitcoin Script used in Tumblebit for one such transaction, on Alice's side; feel free to ignore it if you're not au fait with Bitcoin Script:</p>
<pre>OP_IF<br/> OP_RIPEMD160 h1 OP_EQUALVERIFY<br/> ...<br/> OP_RIPEMD160 h15 OP_EQUALVERIFY<br/> T_pubkey<br/> OP_CHECKSIG<br/>OP_ELSE<br/> locktime<br/> OP_CHECKLOCKTIMEVERIFY<br/> OP_DROP<br/> alice_pubkey<br/> OP_CHECKSIG</pre>
<p>Since these redeem scripts are large (see the h1 .. h15  that's 15x20 byte hashes), they are encapsulated in P2SH addresses</p>
<p>A successful redemption would be revealing the hash preimages, looking something like this:</p>
<pre>OP_0 signature OP_TRUE preimage1 ... preimage15</pre>
<p>The <code>OP_TRUE</code> here tells the script to check the first "IF" condition.</p>
<h3>Blinding</h3>
<p><a href="https://en.wikipedia.org/wiki/RSA_(cryptosystem)">RSA</a> is the oldest of the standard <em>asymmetric </em>cryptographic algorithms, and still in use today. It has a fundamental <a href="https://en.wikipedia.org/wiki/Malleability_(cryptography)">malleability</a>, which is in many ways a weakness, but also is useful. Consider that RSA encryption and decryption operations are just a matter of <strong>exponentiation</strong>: \(E = m^{e}\ \textrm{mod} N\). Here \(m\) is the message to be encrypted, \(e, N\) is the public key and \(E\) is the result; just replace \(e\) with \(d\), where \(d\) is the private/secret key, for decryption.</p>
<p>Such a simple mathematical structure is termed "malleable" because \(E(m_1) \times E(m_2) = E(m_1 \times m_2)\). Whilst this can result in terrible security problems (see e.g. <a href="http://crypto.stackexchange.com/questions/1448/definitionoftextbookrsa">textbookRSA</a>, along with much more advanced variations on the theme), it also creates useful functionality, the most striking example of which is perhaps the idea of the <a href="https://en.wikipedia.org/wiki/Blind_signature">blind signature</a>, first put forward by David Chaum. (Side note: in RSA, signing is basically the same as decryption, i.e. raising a hash to the exponent \(d\) but let's not get lost in the details). A blind signature is allows a central authority to sign data which is <em>hidden from them</em>; although superficially that sounds terrible, it can work well in systems where a user has data (A, B) where A can be verified by the authority while B can be kept private.</p>
<p>Chaum quickly applied the idea to digital cash, and indeed the whole story of "cryptographic money" seems to mostly start with Chaum's ideas, generally called "Chaumian cash", with a central mint authorised to blindsign transfers of this cash, thus keeping privacy for the user. This crystallised in <a href="https://en.wikipedia.org/wiki/DigiCash">digicash</a> in the 1990s (which then famously blew up at least partly due to its centralized model).</p>
<p>This same RSA blinding mechanism can also be used in an encryption, rather than a signing context. Just "compose" encryption with a blinding factor: take an message \(m\), and "blind" it with a random number \(r\), then encrypt it; you get \(E_b = (m r)^{e}\ \textrm{mod}N\), which due to malleability is the same as \(E(m) \times E(r)\). Interestingly (as this will be very important), you don't have to be the owner of the original message to do the blinding: if you're given \(E(m)\) but don't know \(m\), you can still blind it by encrypting your blinding factor with the public key and simply multiplying to get \(E_b\) without first knowing the message.</p>
<p></p>
<h4>How it's used here</h4>
<p>This allows you to delink the encryption and decryption operation performed by a central server \(T\), whose public key you are using. Bob may have \(E(m)\), and wants to know \(m\) but doesn't want to reveal <em>which</em> \(m\) he is decrypting. So he simply generates a random \(r\), and blinds his encrypted message with \(E(r)\) and passes that <em>blinded </em>encryption to \(T\). What \(T\) would pass back as the decryption is \(mr\), and since \(r\) is random, \(T\) gets no information about what he's decrypting. To get \(m\), Bob of course simply divides the result by \(r\) (unlike factoring or root taking, division is not a problem in modular arithmetic, you use the <a href="https://en.wikipedia.org/wiki/Extended_Euclidean_algorithm">Extended Euclidean Algorithm</a> or similar).</p>
<p>In TumbleBit we go further, though: Bob doesn't directly ask \(T\) for the decryption  after first blinding as above, he passes the blinded encrypted value to Alice. It's Alice who then requests the decryption, in "fair exchange" for her payment (see more on this below), and passes back the <em>still blinded </em>decryption \(mr\) to Bob, who then unblinds as before. Introducing this intermediary doesn't create additional problems, but makes it possible for Bob to receive his decryption without revealing what it was a decryption of, which is neat.<em></em></p>
<p>This is the idea that TumbleBit uses to achieve unlinkable payments; in principle the TumbleBit server has no idea who paid who.</p>
<p></p>
<h3>Fair Exchange</h3>
<p>This is the most important part of the system, and so I'm going to make an effort to explain it in <em>some </em>detail. At a very high level, it's using commitments  I promise to have X data, by passing over a hashed or encrypted version, but I'm not <em>yet</em> giving it to you  and interactivity  twoway messaging, in particular allowing commitments to occur in both directions. Folded in is the idea of "cut and choose" (see the paper for references) which makes cheating not literally impossible but vanishingly improbable to achieve successfully.</p>
<h4>The Puzzle Promise Protocol</h4>
<p>You, Bob, want Thomas the tumbler to sign a transaction that pays you money, but you don't want him to know <em>which </em>transaction he's signing. Sounds impossible to do securely for both sides, right?</p>
<p>A brief refresher on how signing works: a transaction is a long list of bytes in a special format, including inputs, outputs and other sundry information. When we ECDSAsign, in order to authorize the transaction on the whole Bitcoin network, we (and this is true for all kinds of digital signatures) don't literally sign the message  in this case the full transaction  but instead sign a hash of it (in Bitcoin it's doubleSHA256, but we'll just call it \(H()\)).</p>
<p>Here's what you do:</p>
<ul>
<li>Bob and Thomas set up an escrow transaction: Thomas puts 1 coin into escrow, redeemable either after a timeout or with the signature of <em>both </em>Bob and Thomas attached. Bob will need this to construct his redeeming transaction, so Thomas passes this transaction across without yet broadcasting it.</li>
<li>You set up a list of "fake" transaction hashes. These will be hashes of <code>FIXED_STRING_FOR_ALL_FAKES123456</code>, where the string is what is says, and the numbers represent some random bytes, different for every case. This list will be long e.g. there will be 42 of them in the current proposal for TumbleBit. The purpose of the random data is to make sure the output of \(H\) is unpredictable (this is called the "hiding" property of a commitment). Call this list \(FH\), fake hashes.</li>
<li>Then, we set up a list of "real" transaction hashes (again, 42 in this particular case); they will all be the same, that's to say, hashes of valid spending transactions, paying me, Bob, 1 BTC, out from one of the escrow type transactions mentioned above. Each one will be random because each one will have a new, freshly generated Bitcoin address as the receiving address, so we again get the "hiding" property here. ECDSA signatures of <em>these</em> hashes will actually authorize spending on the Bitcoin network. Call this list \(RH\), real hashes.</li>
<li>Next, you take these two lists and join them into one list, with the order completely jumbled up. Send the scrambled list over to Thomas. You're sending: \(RH_{11}, FH_{7}, FH_{41}, RH_{32}, \ldots\). (side note: he also sends a commitment to the positions in the list which are fakes/reals, but we'll skip this).</li>
<li>Thomas then ECDSAsigns all of them. At this stage he doesn't know which signatures he's just generated will be real transactions and which not. He's certainly not just going to send the signatures back to you  since then you can spend the ones you know are real, for free! Note that he couldn't figure out which were real from examining the hashes  remember, you used completely random output addresses, so at the moment he just knows he has some ECDSA signatures, all valid, some of which which would be spendable, some not. He can't just check on his Bitcoin node; to broadcast a transaction, you need the actual transaction itself, not just the signature(s)!</li>
<li>Next, Thomas encrypts each of these signatures with a unique encryption key. Here he can just use ordinary symmetric encryption, more specifically he can XOR each signature with a randomly generated key (but forget the details  just encrypts with a secret key only he knows), which we'll call \(\epsilon_i\) where \(i\) runs over the length of the list, in this case from 1 to 84. And we'll call the encrypted signatures \(c_i\). Next, Thomas will RSAencrypt all the secret keys and make a similar list: \(z_i = \textrm{RSA}(\epsilon_i)\).</li>
<li>Thomas will send the list of pairs \(c_i, z_i\) back to Bob at this point. Note how both those values are encrypted (although differently), so Bob has zero information at this point.</li>
<li>Bob will now do a "reveal" step: he tells which 42 out of the 84 he originally sent, were the fakes (he remembered the indexes in his jumbled list, e.g. in the example above index 2 (\(FH_7\)) was a fake). To allow Thomas to verify that they were indeed fake, he sends the specific random "hiding" bytes he used for each fake entry (shown with the "123456" placeholder above).</li>
<li>Given this revelation, Thomas can verify Bob's honesty: he can go through each identified fake commitment and perform \(H(\textrm{FIXED_STRING_FOR_ALL_FAKES}r_i)\) where \(r_i\) is the random padding for that specific fake in the set. This hash should match the one he was originally sent, otherwise Bob didn't follow protocol and is dropped.</li>
<li><strong>Here's the cool thing: because Thomas now knows those entries were fake, he can safely reveal to Bob the decryptions <em>specifically for the now recognized fakes</em>. This is the bit that forced him to honestly and properly ECDSAsign <em>all</em> the original hashes. If he had selectively missigned some of the original set of hashes, he would be relying on probability to "luck out" that the ones he didn't sign properly didn't appear in this list. But probability really works against this kind of fraud, as long as the lists of fakes and reals are long enough: the probability of successful cheating by luck is actually \(\frac{1}{\binom{84}{42}}\), which gives a chance of success of \(\simeq 2^{80}\), or 80 bit security to put it another way</strong>.</li>
<li>So assuming he acted honestly, Thomas can send across the \(\epsilon_i\) secret keys for those fakes. Bob then validates: first, he checks that \(z_i = \textrm{RSA}(\epsilon_i)\). Then, he checks that the signature he gets by decrypting \(c_i\) using that decrypted secret key, validates (he just uses some Bitcoin software for example to do the ECDSAsecp256k1 verification on that signature). He does this for every nowknowntobefake entry in his list, so 42 times. If any of these checks fail, he aborts, as Thomas was not following protocol.</li>
<li>Bob now knows with 80 bit security that the remaining 42 <em>encrypted</em> signatures he possesses are what they claim to be, and the same for the RSAencrypted secret keys (\(z_i\)) that would unlock those signatures.</li>
<li>Thomas broadcasts the escrow transaction now; he knows that Bob will be able to spend out of it if all goes well in the next step, otherwise he (Thomas) will just get the funds back after timeout.</li>
</ul>
<p>You might be wondering what gives with having 42 separate valid real transactions (with asyet undecrypted signatures to go with them). This is an issue in two senses  1, we only want Bob to be able to spend once, but Bitcoin itself, combined with Thomas the tumbler using different public keys for each instance of the protocol, handles the potential doublespend issue, and 2  for the next step we want Bob to be requesting a decryption of just <em>one</em> RSAencrypted key; this part is handled using what is called an "RSA Quotient test"; for "brevity" (hah!), I will omit this, but the details are in the paper.</p>
<p>So how does Bob get hold of at least one of those secret keys (\(\epsilon_i\)) that would enable him to decrypt the signature and broadcast his transaction? <strong>He asks Alice!</strong></p>
<h4>The PuzzleSolver protocol</h4>
<ul>
<li>Bob sends Alice his \(z_1\). First, note this can be on a "sidechannel"; Thomas need know nothing about which Bob talks to which Alice. Second, he doesn't <em>directly</em> send his \(z_1\); he uses blinding! He sends \(y = z_1 \times \textrm{RSA}(r_1)\), so he knows that (if/when) he gets the decryption back, he can unblind it, and neither Alice nor, more crucially, Thomas need know what exactly he was decrypting (see the above section on RSA blinding).</li>
<li><strong>Alice's next set of steps is very similar to what Bob did before</strong>:</li>
<li>Make a list of 15 real puzzles: \(d_i = y \times \textrm{RSA}(r_i)\) where the \(r_i\)s are random; note here we're not hashing, but making RSA encryptions, but just as before we have to "hide" by appending random data.</li>
<li>Make a list of 285 fake puzzles: \(\delta_i = \textrm{RSA}(\rho_i)\), the \(\rho_i\) again being random. The reason for the strange numbers (15, 285) instead of (42, 42) will be explained at the end (see note 1).</li>
<li>As before, Alice scrambles the lists and sends to Thomas, remembering which were real and which fake.</li>
<li>As before, Thomas does the "real" job in secret: in this case, instead of ECDSAsigning something, he's RSAdecrypting all the values sent to him, using the RSA private key which only he possesses. Call these decryptions (the thing Alice wants) \(s_i\), where \(i\) runs from 1 to 300. For each of these, he chooses a new secret symmetric encryption key and encrypts the \(s_i\)s to get \(c_i\)s. He also makes hashes of the secret keys, \(h_i\).</li>
<li>Thomas sends the list of pairs \(c_i, h_i\) to Alice. As before, he's now bound to these decryptions, without revealing them to Alice.</li>
<li>As before, Alice tells Thomas which 285 of the provided 300 were fakes; she sends him the values \(\rho_i\) from above. Thomas checks that, for the specified indices in the list, \(\textrm{RSA}(\rho_i)=\delta_i\) (excuse a slight abuse of notation, of course Thomas had no distinction originally between \(d_i\) and \(\delta_i\)). This part is unobvious but really important: <strong>it checks that Thomas is not going to unwittingly give Alice decryptions as part of the fake set</strong>, after which she could abort the remaining steps and get those decryptions for free. Because Alice was able to provide the preencrypted \(\rho_i\) in its entirety, she already knew the decryption, and it can't have been either \(y\) or \(y \times \textrm{RSA}(r_i)\) (a blinded value).</li>
<li>Thomas, now knowing the fake set, as before, "reveals" the solutions for the fake set: he sends to Alice the secret keys used to encrypt the solutions.</li>
<li>Alice takes each \(c_i\) in the fake set, uses the provided secret keys to decrypt them to \(s_i\) and verifies that they are equal to \(\textrm{RSA}(\rho_i)\) in each case. She also verifies that \(h_i\) is the hash of that secret key. Now she has the same kind of probabilistic certainty of honest behaviour from Thomas that Bob had in the puzzlepromise protocol.</li>
<li>Alice now posts a transaction of exactly the form described in the "Escrow transaction" section above  it pays out either back to her after a timeout, or to Thomas if he provides in the redeeming transaction, the preimages of the hashes \(h_i\) for all 15 in the "real" set (but remember their position in the original list was jumbled).</li>
<li>Alice then is ready to transfer to Thomas the actual blinded puzzle itself: \(y\), as well as the random pads/blinds \(r_i\) for 1 to 15 in the real set.</li>
<li>Thomas verifies that \(d_i = y \times \textrm{RSA}(r_i)\). He can then broadcast a transaction redeeming 1 coin using the 15 hash preimages (what is called above "secret keys"), which will then be visible on the blockchain.</li>
<li>Alice, on seeing this transaction on the blockchain, picks up the hash preimages, and uses them to decrypt to obtain the RSA decryption of \(y\): From \(c_i\), use the secret key to decrypt to \(s_i\), then divide by \(r_i\) to get the RSA decryption of y. (see note 2)</li>
<li>Alice can now pay Bob by passing over the decrypted value (in the simplest model, remember, Alice = Bob); he divides by the blinding factor <em>he</em> used to get the original decryption key \(\epsilon_1\), which he knows from the PuzzlePromise protocol he can use to obtain a valid ECDSA signature on his redeeming transaction, which he then broadcasts. Done!</li>
</ul>
<p>Notes:</p>
<ol>
<li>Why 15/285? This is because the key transaction at the end, in which Thomas broadcasts the hash preimages, which are secret keys, for Alice to read, must be valid on the Bitcoin blockchain, and it cannot contain hundreds of secret key values; for 128 bit keys, 16 bytes are used per key, so 15 of these requires 240 bytes; if the number was much larger, it would not be feasible. So to ensure 80 bit security, we need \(1/\binom{m+n}{n} \simeq 2^{80}\), and if \(n=15\), this means \(m\) must be much larger, around 285 here.</li>
<li>Note that it's vital he checks that \(y\) is the same for all the "real" cases; otherwise he could be unwittingly providing <em>more</em> than one puzzle solution to Alice for only 1 coin! As mentioned earlier, Alice cannot "cheat" this because she is unable to create RSA encryptions, even with blinding, without knowing the original message (a bit like the inability to create hash preimages).</li>
</ol>
<p></p>
<h3>Final thoughts  privacy, scalability, practicality</h3>
<ul>
<li>RSA operations are notoriously expensive; for a classic tumbler I can't imagine a realistic scenario where this will matter. It <em>might</em> matter for a payments hub model in which a lot of (especially offblockchain) transaction events are happening; the paper gives some performance data on this.</li>
<li>The classic tumbler model assumes fixed denominations; I doubt that this could be avoided in such a model, although I haven't thought about it much. The effect of things like 800 in  800 out transaction sets of fixed size, and with proper delinkage, is so powerful though, and combined with the centralizationforcoordinationbutnotrequiringtrust, might well mean that this problem is offset. One could imagine just doing a "mix" of a fixed chunk of one's funds at regular intervals, for example. But more on "coordination":</li>
<li>The paper asserts that TumbleBit requires "no coordination" (Table 1) in contrast to other models requiring peertopeer coordination. This is true in one sense, but I think a little misleading, since all parties have to agree on an amount size and a time to do the mix  not with each other, but with the TumbleBit server. This is in contrast to Joinmarket, where amounts are set by the user of the service, and liquidity is provided by passive waiting ("Makers") by those who expect to be rewarded for that. I think the balance is in TumbleBit's favour, compared to Joinmarket and other Coinjoin coordinating protocols, but the proof of the pudding is in the eating!</li>
<li>Scalability: can be considered in two contexts: in the context of <em>mixing</em> in particular, this is potentially a huge win over other untrusted models, like Coinjoin, where Bitcoin transaction size limits a set of counterparties to 1020 realistically. This is a gain of 12 orders of magnitude. For the general use of Bitcoin scalability, it might be a model that helps but only in the same sense as other payment channel solutions, so again, I'm keeping that out of scope.</li>
<li>Network level anonymisation: Alice and Bob will likely be one person in a classic tumbler mode, so if that's the case, it's important that the roles "Alice" and "Bob" are not just dumbly coming over the same naked IP address, since then we lose the property of "unlinkability to the tumbler server". Theoretically this is just a detail, but it might be important to consider in practice, from a usability perspective.</li>
<li>When trying to achieve privacy you need to avoid both amount and timing correlation. The amount part is implicit in the current scheme; the timing part should be OK too; the payins on Alice's side and the payouts on Bob's side are separate, so I suspect with a little bit of design care one can avoid the situation where the same party acting as Alice and Bob is acting at the "same" time in such a distinguishable way that one could easily see who is acting as Alice_1 and Bob_1, so to speak.</li>
</ul>
<p></p>
<p></p>Racing against snoopers in Joinmarket 0.220160805T11:30:52+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/racingagainstsnoopersinjoinmarket02/<p>In my <a href="https://joinmarket.me/blog/blog/joinmarket020changes/">previous</a> post, I talked about the two most important new functionalities in the 0.2 version that's currently in <a href="https://www.reddit.com/r/joinmarket/comments/4vm2h9/volunteers_for_testnet_testing_of_020_version/">testing</a>. It's particularly the second of those two, the defence against privacy degradation via snooping on utxos, that's important, and most in need of thought, analysis and explanation.</p>
<p>The more I think about it, the more it seems that this can only be properly understood in a wider context, so I'll, here, attempt to answer these questions:</p>
<ul>
<li>What is an attacker trying to achieve?</li>
<li>How does he achieve it now (if indeed he does)?</li>
<li>How does the new functionality in 0.2.0 make it more difficult to achieve?</li>
<li>What other measures can or should be taken to make it harder?</li>
</ul>
<h3>Preamble  a reminder of the basic Joinmarket <em>conversation</em></h3>
<p>This is how the conversation went pre0.2, and it will be mostly unchanged; note that this is happening simultaneously for several Makers (and one Taker):</p>
<table border="4" cellpadding="1" cellspacing="1" height="373" style="bordercolor: black; height: 373px;" width="713"><caption></caption>
<tbody>
<tr>
<td><span style="textdecoration: underline;"><strong>Taker</strong></span></td>
<td></td>
<td><span style="textdecoration: underline;"><strong>Maker</strong></span></td>
</tr>
<tr>
<td>I'd like to fill your offer number 0 (which allows any amount between 0.5 and 5btc) to do a coinjoin, I want amount 1btc [fill]</td>
<td>>>></td>
<td></td>
</tr>
<tr>
<td></td>
<td><<<</td>
<td>OK, here's my ECDH pubkey [pubkey]</td>
</tr>
<tr>
<td>OK, here's mine [auth] **</td>
<td>>>></td>
<td></td>
</tr>
<tr>
<td colspan="3" style="textalign: center;">FROM NOW ON THE CONVERSATION IS E2E ENCRYPTED</td>
</tr>
<tr>
<td></td>
<td><<<</td>
<td>Here's a list of my utxos for your transaction [ioauth]**</td>
</tr>
<tr>
<td>
<p>(Builds transaction after getting utxos from all Makers).</p>
<p>OK, here's the transaction, please sign it [tx]</p>
</td>
<td>>>></td>
<td></td>
</tr>
<tr>
<td></td>
<td><<<</td>
<td>Here's my signature(s) [sig]</td>
</tr>
<tr>
<td>
<p>(gathers all the signatures)(adds his own signatures)</p>
<p>Broadcasts transaction to Bitcoin network</p>
</td>
<td></td>
<td></td>
</tr>
</tbody>
</table>
<p>**  this misses some details that aren't vital for this discussion.</p>
<p>The words in [] are the names of the messages, some of which are referred to in the discussion below.</p>
<h2>What is the attacker trying to achieve?</h2>
<p>We'll focus on the, most likely, frequent goal: if an attacker already knows the source of the inputs, he wants to identify the coinjoin output of the initiator (Taker) of the transaction (figuring out the change output is always easy). Here's a simple diagram of a basic Joinmarket coinjoin transaction, nabbed from <a href="https://github.com/AdamISZ/JMPrivacyAnalysis/blob/master/tumbler_privacy.md#joinmarkettransactiontypes">here</a> :</p>
<p><img alt="jmtx1" height="372" src="https://joinmarket.me/static/media/uploads/cjmtx.svg" width="526"/></p>
<p>Remember, the aspect of CoinJoin that Joinmarket leverages, as illustrated in the "coinjoin outputs" above, is that those equal sized outputs are indistinguishable. The idea is that, even if you know the ownership of the inputs (utxo0,..4 above) (which you might not), it doesn't help you figure out the ownership of utxos 5,6 and 7.</p>
<p>Now, the attacker doesn't just look at one transaction  he looks at all the Joinmarket transactions, of which this is just one. Suppose the initiator (Taker) of the transaction spent utxo1 to utxo5 and utxo8 (fees are ignored in this example). That would mean that utxo6 and utxo7 belong to the other participants  the "Makers" in joinmarket. The attacker can prove this as follows:</p>
<ul>
<li>Query the makers in the joinmarket pit using <code>fill</code> after a few handshake negotiation messages, the maker will return an <code>ioauth</code> message, which lists the utxos that the Maker proposes to include in the new transaction. Now the attacker has no intention of going through with it; he just wanted that list of utxos. Depending on <em>stuff</em> (we'll get to this below), this list (it looks like txid:n, txid:n .. ) <strong>might</strong> include one of the outputs as above, for example utxo6. If so, he crosses off utxo6 on his diagram above; he knows that one wasn't the one that belonged to the Taker.</li>
<li>Who should he query? If the pit is full it might be quite a lot of work to query all of them (say 70100) all the time. Before 0.2.0, though, he can do this, and repeatedly, and quickly. But he can be more intelligent: focus his efforts on Makers that either (a)have just joined (remember: bots have ephemeral names, a newly arrived bot may simply be an old one with a new name) or (b)have newly reannounced their orders, something that they <em>sometimes</em> do after a transaction.</li>
<li>How often should he query? This turns out to be a really important question. First, doing only one query means he has to decide what amount to query with (the <code>fill</code> message has to specify the amount of coins that you want to join). Mostly he'll try to query with the maximum amount, so as to get the list of all utxos, but different Maker bots have a variety of weird ways of specifying their offers; sometimes they offer all coins in one mixdepth, sometimes it's a complex mixture. He might find that he needs to do multiple queries to the same bot to get the maximum possible snooping effect, and in some cases it might not be enough. Second, he cannot be <em>too </em>slow in doing these queries, for the important reason that the utxos he's looking for might get used in <strong>new</strong> transactions, thus getting consumed, and will no more be offered in ioauth messages from that Taker. At that point he has <strong>permanently lost</strong> the opportunity to identify the Maker as the holder of that utxo.</li>
</ul>
<h3>How does he achieve it now (if indeed he does)?</h3>
<p>So let's paint the positive picture for the attacker, in Joinmarket pre0.2. He gets to make these queries as often as he likes, subject only really to the limitations of the messaging channel (the IRC server). That means, very often, if he feels like it. His bots can be restarted as often as he likes, he can connect over Tor  there is no way to flag his identity and ban him, <em>for exactly the same reason that Joinmarket users appreciate having an anonymised messaging service</em>. Most Joinmarket users strongly favour using Tor for this same, good reason. Note, further, that his actions up to the point of dropping the transaction are <em>exactly</em> the actions of an honest participant. So he does tons of request>utxo collection pings on all the Makers he wants, perhaps a few for each, to get a good chance of hoovering up within his net, the utxos that he is looking for (I have thus far emphasised the coinjoin outputs, they're critical, but of course he is also collecting utxos that will appear as inputs in the <em>next </em>transactions  often they will be the same, plus previous change). If he manages to hoover the right ones up <em>before </em>they get consumed in a new transaction, he's done his job  he knows, say, that utxo6 belongs to maker A and utxo7 belongs to maker B and that by elimination, utxo5 was the output of the Taker.</p>
<p></p>
<p>If you've got the basic gist from that description (the exact details are fiddly and in some sense beside the point), then this quote of mine from the <a href="https://github.com/JoinMarketOrg/joinmarket/issues/156#issuecomment123311971">original discussion</a> may resonate:</p>
<blockquote>The fundamental issue is that makers <strong>announce</strong> their utxos to anyone who (pretends to) start a transaction with them; whether this can be avoided without creating hideous DOS issues I don't know.</blockquote>
<p>So, given that using any kind of identities is not an acceptable solution, the only way we can stop this is by taking a combination of two approaches:</p>
<ol>
<li>Make the ability to gather utxos in a transaction request as limited, or unreliable as possible (<strong>yield generator coin selection algorithms</strong>)</li>
<li>Make the ability to gather utxos rate limited by enforcing some kind of scarcity (<strong>taker commitment schemes</strong>)</li>
</ol>
<p></p>
<p>We'll take these in reverse order, and then see the overall effect on an attacker.</p>
<p></p>
<h2>How does the new functionality in 0.2.0 make it more difficult to achieve?</h2>
<p>The mechanism in 0.2.0 has been covered in depth in <a href="https://joinmarket.me/blog/blog/poodle/">these</a> <a href="https://joinmarket.me/blog/blog/joinmarket020changes/">posts</a>. It was also mentioned that there are other possibilities, so let's treat the high level concept of "utxos are limited". Now, the Taker (who may be an attacker) does not, in the conversation sequence shown in the preamble, have to provide a utxo (actually he did for another reason but that is off topic so ignored), so this is a new field in the message; instead of</p>
<pre>!fill offerid amount</pre>
<p>we switch to</p>
<pre>!fill offerid amount commitment=(type byte,hash(pubkey2)) nicksignature</pre>
<p>The significance of <code>nicksignature</code> is discussed in the previous blog post and so ignored here. The commitment is a SHA256 hash of the public key corresponding to a paytopubkeyhash (P2PKH), i.e. standard, Bitcoin utxo (so technically it's a commitment to a Bitcoin keypair, which is a bit more restrictive than a utxo, although hopefully most people are not reusing addresses too much!), derived from PoDLE as previously described. The commitment is prepended with a type byte, in this case "P", for future compatibility with other commitment types.</p>
<p>The utxo committed to is required to have certain key properties as described in the section <strong><a href="https://joinmarket.me/blog/blog/joinmarket020changes/">Adding commitments to slow down snoopers</a></strong> . If the commitment doesn't pass the test of nonreuse, the Maker drops the Taker right after the fill message. If the commitment is new, but on opening of the commitment  which in the new 0.2 protocol comes in the auth message:</p>
<pre>!auth commitmentopening=(utxo,pubkey,pubkey2,schnorrsig) nicksignature</pre>
<p> it is found to not pass those tests, then again the Taker is dropped  <em>before</em> the ioauth message is sent, thus the Taker, if an attacker, does not achieve his goal of utxo list gathering.</p>
<p></p>
<h3>Quantitative guesstimates</h3>
<p>How many utxos, and of what type does an attacker need? Let's imagine some numbers (these represent a bigger, more active Joinmarket than today, but only by a smallish factor):</p>
<p>Number of makers in the pit \(N_m = 100\), number of makers in one transaction \(N_t = 3\) (note that's 4 counterparties including Taker in a typical Joinmarket transaction, about right). That means there are 3 coinjoin outputs in a transaction that the attacker is trying to "place". Number of transactions per hour \(\alpha = 20 \). Number of queries to <em>one </em>maker needed by the attacker to retrieve the intended utxo, if indeed it can be retrieved: \(Q = 2\). Success probability after \(Q\) queries: \(p = 0.8\). In the crudest model you'd consider the attacker needing to query <em>every </em>maker in the pit after every transaction occurs, which would need: \(\beta = \alpha N_m Q= 4000\) separate commitments per hour. Given <code>taker_utxo_retries</code> = 3 (see previous blog post), that would mean ~ 1000 utxos needed, satisfying age requirements in each hour, and perhaps more importantly, their size has to be at least <code>taker_utxo_amtpercent</code> percent (default 20%) of the largest available size in the offers, typically  so even if a maker just did a transaction of only 1 btc, if his offer is 10 btc, your attacker utxo commitment still needs to bind to a commitment of 2 btc all the same, in order to pass the auth check in that case.</p>
<p>Now, here's a big reason that this is, currently, unrealistic: <em>Most</em> Makers today<em></em> reannounce their offers immediately after doing a transaction (they basically make adjustments because the number of coins on offer is changing etc.). That gives the attacker a big headstart: he can focus on querying those reannouncements, and if he's successful, stop there (for that utxo). In the worst case you could imagine only needing \(\beta = \alpha N_t Q = 120\) instead of 4000. The truth may be between extremes, accounting for e.g. bot restarts and a host of other factors.</p>
<p>One could easily imagine, that to fulfil commitment requirements, the attacker will need to generate utxos holding maybe even in the hundreds of BTC (remember: 20% restriction on large fills), refreshing them into new utxos by doing transactions every hour (the 1 hour figure is particularly relevant since by default we have chosen <code>taker_utxo_age</code> to be 5 blocks). They don't have to spend them, but they have to keep cycling them in transactions (and an amusing detail is: they can be tracked by the honest Joinmarket participants  that'd make for a curious role reversal..).</p>
<p>But, does the attacker need to have fresh utxos satisfying these criteria <em>every</em> hour, even under these circumstances?</p>
<p></p>
<h3>The race</h3>
<p>The above guesstimate scenario is so full of wild assumptions as to be borderline ridiculous, but we have to start somewhere. We find an important dynamic going on: the attacker has to try to keep up with pit activity. If he does the ~ 4000 queries as outlined above in one hour, and then takes a rest for a few hours, he is failing in his goal (if his goal is to identify the ownership of <em>all</em> outputs, anyway): this is a job that can't be done retroactively. Once a utxo is generated in transaction 1, and then used later in transaction 2, it will never again appear in any Maker's offer, so it cannot be identified using this attack any more. So we see there is a <strong>race going on between the attacker and the honest pit </strong> the faster the real Joinmarket transactions are taking place, the faster the attacker has to query Makers in order to catch utxos <em>between</em> transactions. In the simplest possible mathematical terms, \(\beta \propto \alpha\)  there's a linear proportionality between real transaction rate and commitment requirement for snoopers, no matter what are the right values for other variables. And since utxos of certain size and age are fundamentally scarce, this is good news. In the limit, one can imagine an optimistic future with even 100 joinmarket transactions per individual block, making the attacker's job here nigh on impossible.</p>
<p>It seems entirely possible to build a <em>fairly</em> sophisticated mathematical model of the dynamics of this race; but we'll leave that for another time.</p>
<h2>What other measures can or should be taken to make it harder?</h2>
<p>Apart from the ratelimiting element, the other crucial element is how reliably and frequently does a request for utxos result in receiving the utxos that the attacker is looking for? This would address the values \(Q\) and \(p\) in the above. We want to increase the former and decrease the latter as much as possible. I see several ways to address this from within the algorithms, and patterns of behaviour of the Makers. To <a href="https://github.com/JoinMarketOrg/joinmarket/issues/156#issuecomment123750171">quote Chris Belcher</a> in the same discussion as above:</p>
<blockquote>
<p>So maybe part of the solution is a <strong>polyculture of maker algorithms</strong> ?</p>
</blockquote>
<p>Let's make a list again!</p>
<ol>
<li>Introduce randomization into the utxos that a Maker selects for a particular transaction  will tend to increase the number of requests to find <strong>that</strong> utxo.<strong><em></em></strong></li>
<li>Prefer algorithms that restrict the offers to one mixdepth; as was the case for the first, simplest one coded: <strong>yieldgeneratorbasic</strong>. This is maybe the least fun idea but an extremely important one in my opinion: limit and isolate the utxos on offer at any particular time, and in particular if it so happens that that set does <strong>not</strong> include the utxo the attacker is looking for, the game is <em>temporarily </em>won  but it's complex because at some point in the future, that utxo is going to be reannounced as available for joining usually  what's clear is it makes the attacker's job much harder if this doesn't happen quickly. The tension we're trying to address is that the Makers are trying to maximize their marketability (e.g. offering a variety of different cost rates for different utxos), while at the same time preserving privacy, but if we are more aggressively "privacy preserving" like this, it makes the whole system much more viable ... slight tragedy of the commons effect.</li>
<li>Restart bots frequently  this simple measure forces the attacker to make more queries, although it is helpful if offer announcement does not make it immediately obvious that it's the same bot as before!</li>
<li>Don't make the algorithm of the maker obvious directly from it's public pit offer announcements; that makes the attacker's job easier since he can fine tune his fill requests based on knowing the code/algorithm in advance.</li>
<li>Prefer algorithms that <strong>don't</strong> require reannouncement of offers after transaction completion, as this is a very useful marker for an attacker as to who to query.</li>
<li>More exotic ideas  crossmaker utxo consumption e.g. maker does "real" Joinmarket transaction, then switches role to Taker, does another join with the other makers quickly using up the utxo (to be honest I find this idea, which I just made up, entirely confusing and maybe nonsensical, but just throwing it out there)</li>
<li>More, no doubt...</li>
</ol>
<p></p>
<h3>Conclusion</h3>
<p>In summary, the measures described in the previous 2 sections, in some reasonable combination, may greatly reduce the privacy loss we are seeing from an unlimited attackerquerier environment; although there's no reason to think such attacks would stop entirely. The first of the two (utxo rate limiting) is now basically complete and running on testnet. The latter is not really (although just running ygbasic is a good start).</p>
<p>But what I think is more significant is, as mentioned in the subsection "The Race", <strong>the more Joinmarket scales up in size, the more difficult it gets to make these attacks work.</strong> In the extreme, if Joinmarket scaled up by a factor of 10 to say 700 Makers and 50 transactions per hour (completely made up numbers, like all the rest!), the attack might become almost completely ineffective.</p>
<p></p>
<p></p>
<p></p>
<p></p>Joinmarket 0.2.0 changes20160801T19:27:17+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/joinmarket020changes/<h2>Two big changes in Joinmarket</h2>
<p>A somewhat unreadable summary of the new protocol can be found <a href="https://gist.github.com/AdamISZ/baf93ce2589854a7992383b3c69fae13">here</a>.</p>
<p>If you haven't used Joinmarket, this is going to be hard to understand; let me suggest browsing <a href="https://github.com/JoinMarketOrg/JoinMarketDocs/blob/master/Highleveldesign.md">this design doc</a> , especially the sections on "Transactions" and "Entities", although if this is 100% new to you, you probably need to start with the <a href="https://github.com/JoinMarketOrg/joinmarket/blob/master/README.md">joinmarket main page</a> and leave this till later. At the very least, know that in Joinmarket a "Maker" is an entity that sits in the joinmarket pit, offering to do joins for a price, while a "Taker" is an entity that takes up these offers and constructs the coinjoin transaction, paying the fees for the privilege of defining the size, being the coordinator and getting the transaction immediately.</p>
<h3>Multiple messaging servers</h3>
<p>First, simpler conceptually but quite important is: <em>running on multiple message channels</em>. Currently that just means multiple IRC servers, but it is possible for Joinmarket to run on other messaging servers; for example, I've tried it on <a href="https://www.matrix.org">matrix</a>; it works, although the behaviour is not quite viable for now. There may well be other good alternatives, including people setting up their own servers. The code now allows this in a straightforward manner, by subclassing <code>messagechannel.MessageChannel</code>.</p>
<p></p>
<p>This buys us a lot. First, we can avoid the frequent problem of a single IRC server going down and bringing all transactions to a halt. Second, it gives an extra layer of censorship resistance  if, as intended, Maker bots make themselves available on multiple channels, there isn't the possibility of a sneaky server operator preventing counterparties seeing their offers. Offers that are published on 3 servers are treated just the same as if they are published on only 1 (assuming the Taker is connected to all of them, of course).</p>
<p></p>
<p>Potential problems with this, and solutions:</p>
<p>The code allows automatic switchover between messaging servers if a connection gets cut during a transaction (it's not perfect in this regard but there are test cases that show it can work, at least a lot of the time, which is a heck of a lot better than nothing). But since a bot is not a persistent identity, this throws up the potential issue of <em>spoofing</em>  if my bot is called "waxwing" on server A and B, what's to stop someone else registering/using "waxwing" on server C, should I not currently be connected to C? To be clear, successful spoofing is mostly limited to "order stealing" (pretend you're waxwing and nab the transaction from me) and DoS effects, since the private data is communicated with E2E encryption already, but that's still something we have to avoid.</p>
<p>To address this, we give the bots an <em>ephemeral</em> pseudonymous identity for the period that they are running  they generate a keypair and, much like <a href="https://gitweb.torproject.org/torspec.git/tree/rendspec.txt#n526">Tor hidden services</a>, their name is tied to their public key with a simple encoding (in this case, truncated base58 of hash of pubkey). Then every private message has a signature tagged onto the end, that verifies only to this pubkey. These signatures are attached to every message, E2E encrypted or not. Meaning, once the conversation starts, it can continue on any message channel, but only with the owner of that ephemeral keypair.</p>
<p>This has an extra little hole at the beginning of the conversation  since the initial message from the Maker (the order announcement) is in plaintext and already published in the "pit" channel, the attached signature is replayable on another server. Such replay is avoided by binding the signature to the server on which it operates by adding a tag into the signed message which indicates the server (see the <code>hostid</code>field in the config file). This patches up the anti"order stealing" defence mentioned above. It might be possible to tighten this defence further; remember that for the initial nonE2E encrypted portion of the conversation, the server has control anyway.</p>
<p>The other issue with this signing approach is rather technical, but to mention it: adding signatures pads out the channel messages with quite a bit of data, but it's not turning out to be a big problem. Currently both the pubkey and the signature are appended, which is wasteful; we can use ECDSA public key recovery, but that can be added in a later update, as there are some rather fiddly details to work out.</p>
<p></p>
<h3>Adding commitments to slow down snoopers</h3>
<p>In my <a href="https://joinmarket.me/blog/blog/poodle/">previous post</a> I described the basic idea of PoDLE. This is implemented in 0.2.0. At a high level, it functions as a rate limiter, based on the fact that utxos are somewhat scarce in Bitcoin. The cryptographic trick in PoDLE means that we can require a joinmarket user to commit to a utxo without revealing it in advance, and so not losing privacy just by <em>suggesting </em>a transaction. If they want to get access to Makers' utxos, they need to "use up" (reveal, and not use again more than a certain number of times  <code>taker_utxo_retries</code> in the config) their utxos. We ratchet up the scarcity with further config variable <code>taker_utxo_age</code> and <code>taker_utxo_amtpercent</code>, restricting them to use only utxos of a certain age and certain size relative to the size of the proposed coinjoin. These values are relatively lax and not intended to be changed by users initially  general, although not complete, agreement on these values will be needed or at least highly preferable. The intention of the code as it stands is to give the honest Taker the maximum chance of being able to do a transaction with no intervention  by choosing utxos from this transaction, or otherwise elsewhere from his wallet  while making it difficult for a snooper to carry out the <a href="https://gist.github.com/chrisbelcher/00255ecfe1bc4984fcf7c65e25aa8b4b">privacy degrading attack on Joinmarket which we've been suffering</a>. There is a lot more to say about this attack, how to defend against it, but I'll defer that for a later post and concentrate on the functionality in 0.2.0.</p>
<p></p>
<ul>
<li><code>taker_utxo_retries</code>  this is accomplished with a nice trick that extends what was in the previous PoDLE <a href="https://joinmarket.me/blog/blog/poodle/">blog post</a>. Greg Maxwell pointed out that multiple retries can easily be achieved by choosing multiple NUMS values in a deterministic way  for those interested, the code I've implemented to do it is <a href="https://github.com/JoinMarketOrg/joinmarket/blob/8f50282a995501449f2a9404a3928c30c5467e85/bitcoin/podle.py">here</a>. This means that the hash value for a utxo depends on an integer selected; so if <code>taker_utxo_retries=3</code>, the Taker will choose one from 0,1,2 and generate the hash for that value. This means that each commitment hash is only allowed to be used once, but the same utxo can be used 3 times. This is enforced on both sides, more on that in a minute.</li>
<li><code>taker_utxo_age</code>  self explanatory; check if the utxo has >= \(N\) commitments. I consider this to be a very important element, because if very small (say 1), an attacker's only requirement is to have a large number of utxos and simply "regenerate them"  say he has 10 utxos, spend them to 10 new utxos. It costs fees, and it must wait on a confirmation, but that isn't very long. More on why this is important in a later post.</li>
<li><code>taker_utxo_amtpercent</code>  if an honest taker wants to do a 1BTC coinjoin, it isn't unreasonable to expect them to have a 0.2 btc (20%) utxo "hanging around" to use  most likely one of the inputs for their transaction, or in the wallet, or outside. Meanwhile a snooper trying to get the whole orderbook might need a lot of fairly large utxos to get the top end of the orderbook with this restriction. So, it's considered very useful, although note of course they don't have to <em>spend</em> these coins (except to themselves to regenerate).</li>
</ul>
<p>Potential problems and solutions</p>
<p>What happens if the Taker doesn't have such a utxo lying around? The "retries" part will only fail if he's already tried to use it \(N\) times (3 times by default), and if he has no other utxos that fit. The "age" part is something that could easily happen with a newly loaded wallet, if we use the default of 5 confirmations. Still, waiting an hour is not the end of the world  and it's only likely to happen on first use of a wallet. The "amtpercent" filter is also quite unlikely to cause a problem.</p>
<p>Still, all of these cases can happen, and it would be a shame to leave a Taker unable to use the system in certain cases. The solution provided is to add utxos external to the wallet  this is messy, not just because it's inconvenient, but mainly because it requires access to the private key for those external utxo(s). Still, a tool is <a href="https://github.com/JoinMarketOrg/joinmarket/blob/8f50282a995501449f2a9404a3928c30c5467e85/addutxo.py">provided</a> that will allow exactly that.</p>
<p>These commitments are stored by the taker in a file <code>commitments.json</code>. In most cases it's hoped the Taker (Joinmarket customer, let's say) won't have to look at it; it'll store what utxos have already been used, as well as any external utxos that are being stored for usage. So they should not delete it.</p>
<p>On the Maker side meanwhile, a file <code>blacklist</code> (unfortunate name perhaps!) stores very simply the hashes (remember \(H(P2)\) from the <a href="https://joinmarket.me/blog/blog/poodle/">PoDLE post</a>) of the commitments used by other Takers. Given the privacy preserving property of these values, they could be freely shared between Makers if they like; <span style="textdecoration: linethrough;">no broadcast method has been implemented yet, but it could easily be done if that's deemed necessary/useful</span> This has now been done, and is switched on by default. Note that there is no way to maliciously broadcast such commitments; you cannot guess the correct hash value(s) for a particular utxo. As for the blacklist file, the Maker can leave it accruing values indefinitely, and probably should, but it won't be a disaster to add random numbers to it, or delete some or all of it; it just means repeat usage of utxos is being allowed.</p>
<p>The real problem with the system is the potential for troublemaking Makers to simply drop transactions of honest Takers, and thus force them to use up more and more of their utxos \(\times\) retries. Whilst this is a problem, note that this kind of DOS already exists, if any Maker wants to do it. The existing pre0.2.0 code already had the facility to restart and ignore a nonresponsive Maker, and that persists. Slightly more failures are expected, and the new user will have a higher chance of getting blocked out  they may only have 1 or 2 utxos available, so only 3 or 6 chances to get the transaction through without having to mess around sourcing external utxos.</p>
<p>Due to this concern I think it's important that (a) the limits are fairly lax  3 retries, 5 confirms, 20% amount requirement and (b) all the users do *not* just randomly reset these values if they feel like it. Much better if we all use "standard" values, albeit of course a few will choose to be more strict.</p>
<p></p>
<p>The benefit.</p>
<p>Will the snooper be stopped by this? Possibly initially, possibly not  and ultimately the snooper <em>cannot </em>be completely stopped, just restricted. In the next post I will talk about the attack in more depth, and also about how yield generators can operate to give further defence against this attack.</p>P(o)ODLE20160622T19:21:21+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/poodle/<blockquote><em>Here is a purse of monies ... which I am not going to give to you.</em></blockquote>
<p> <a href="https://en.wikipedia.org/wiki/Bells_(Blackadder)">Edmund Blackadder</a></p>
<p><img alt="" height="308" src="https://joinmarket.me/static/media/uploads/.thumbnails/poodle.jpeg/poodle225x308.jpeg" width="225"/></p>
<h3>P(o)ODLE, not POODLE</h3>
<p>This post, fortunately, has nothing to do with faintly ridiculous <a href="https://en.wikipedia.org/wiki/POODLE">SSL 3 downgrade attacks</a>. Irritatingly, our usage here has no madeup need for the parenthetical (o), but on the other hand "podle" is not actually a word.</p>
<p></p>
<h3>The problem</h3>
<p>You're engaging in a protocol (like Joinmarket) where you're using bitcoin utxos regularly. We want to enforce some scarcity; you can't use the same utxo more than once, let's say. Utxos can be created all the time, but at some cost of time and money; so it can be seen as a kind of rate limiting.</p>
<p>So: you have a bitcoin utxo. You'd like someone else to know that you have it, <strong>and that you haven't used it before, with them or anyone else</strong>, <strong>in this protocol,</strong> but you don't want to show it to them. For that second property (hiding), you want to make a <em>commitment </em>to the utxo. Later on in the protocol you will open the commitment and reveal the utxo.</p>
<p>Now, a <a href="https://en.wikipedia.org/wiki/Commitment_scheme">cryptographic commitment</a> is a standard kind of protocol, usually it works something like:</p>
<p></p>
<pre>Alice>Bob: commit: h := hash(secret, nonce)<br/>(do stuff)<br/>Alice>Bob: open: reveal secret, nonce<br/>Bob: verify: h =?= hash(secret, nonce)</pre>
<p></p>
<p>Hashing a secret is <em>not</em> enough to keep it secret, at least in general: because the verifier might be able to guess, especially if the data is from a smallish set (utxos in bitcoin being certainly a small enough set; and that list is public). So usually, this protocol, with a largeenough random nonce, would be enough for the purposes of proving you own a bitcoin utxo without revealing it.</p>
<p>But in our case it doesn't suffice  because of the bolded sentence in the problem description. You could pretend to commit to <em>different </em>utxos at different times, simply by using different nonces. If you tried to do that <em>just with me</em>, well, no big deal  I'll just block your second use. But you <em>could </em>use the same utxos with different counterparties, and they would be none the wiser, unless they all shared all private information with each other. Which we certainly don't want.</p>
<p>Contrariwise, if you ditch the nonce and just use Hash(utxo) every time to every counterparty, you have the failuretohidethesecret problem mentioned above.</p>
<p>In case you didn't get that: Alice wants to prove to Bob and Carol and ... that she owns utxo \(U\), and she never used it before. Bob and Carol etc. are keeping a public list of all previously used commitments (which shouldn't give away what the utxo is, for privacy). If she just makes a commitment: Hash(\(U +\) nonce) and sends it to Bob and Carol, they will check and see it isn't on the public list of commitments and if not, OK, she can open the commitment later and prove honest action. But her conversations with Bob and Carol are separate, on private messaging channels. How can Bob know she didn't use <em>the same utxo as previously used with Carol, but with a different nonce</em>?</p>
<p></p>
<h3>The solution</h3>
<p>This is a bit of a headscratcher; after several IRC discussions, Greg Maxwell suggested the idea of <strong>proof of discrete logarithm equivalence </strong>(hence the title), and pointed me at <a href="http://crypto.stackexchange.com/questions/15758/howcanweprovethattwodiscretelogarithmsareequal">this crypo.stackexchange thread</a>. It's a cool idea (although note that that description is based on DL rather than ECDL seen here): "shift" the EC point to a new base/generator point, so that nobody else can read (crudely put), then append a Schnorr signature acting as proof that the two points have the same discrete logarithm (= private key) with respect to the two base points. In detail, consider a Bitcoin private, public keypair \((x, P)\) for the usual base point/generator \(G\), and consider a <a href="https://en.wikipedia.org/wiki/Nothing_up_my_sleeve_number">NUMS</a> alternative generator \(J\) ( a little more on this later).</p>
<p>$$P = xG$$</p>
<p>$$P_2 = xJ$$</p>
<p>Next, Alice will provide her commitment as \(H(P_2)\) in the handshake initiation stage of the protocol. Then, when it comes time for Alice to request private information from Bob, on their private message channel, she will have to open her commitment with this data:</p>
<p>$$P, U, P_2, s, e$$</p>
<p>Here \(s,e\) are a Schnorr signature proving equivalence of the private key (we called it \(x\) above) with respect to \(G,J\), but of course without revealing that private key. It is constructed, after choosing a random nonce \(k\), like this:</p>
<p>$$K_G = kG$$</p>
<p>$$K_J = kJ$$</p>
<p>$$e = H(K_G  K_J  P  P_2)$$</p>
<p>$$s = k + xe$$</p>
<p>Then Bob, receiving this authorisation information, proceeds to verify the commitment before exchanging private information:</p>
<ol>
<li>Does \(H(P_2)\) equal the previously provided commitment? If yes:</li>
<li>Check that the commitment is not repeated on the public list (or whatever the policy is)</li>
<li>Verify via the blockchain that \(P\) matches the utxo \(U\)</li>
<li>\(K_G = sG  eP\)</li>
<li>\(K_J = sJ  eP_2\)</li>
<li>Schnorr sig verify operation: Does \(H(K_G  K_J  P  P_2) = e\) ?</li>
</ol>
<p>Bob now knows that the utxo \(U\) has not been repeated (the simplest policy) but Alice has not been exposed to a potential public leakage of information about the utxo. (It should be noted of course! Bob knows the utxo from now on, but that's for another discussion about Coinjoin generally...)</p>
<p></p>
<h3>Why an alternate generator point \(J\)?</h3>
<p>Publishing \(H(P_2)\) gives no information about \(P\), the actual Bitcoin pubkey that Alice wants to use; in that sense it's the same as using a nonce in the commitment. But it also gives her no degree of freedom, as a nonce does, to create different public values for the same hidden pubkey. No one not possessing \(x\) can deduce \(P\) from \(P_2\) (or vice versa, for that matter)  <strong>unless</strong> they have the private key/discrete log of \(J\) with respect to \(G\). If anyone had this number \(x^*\) such that \(J = x^{*}G\), then it would be easy to make the shift from one to the other:</p>
<p>$$P_2 = xJ = x(x^{*}G) = x^{*}(xG) = x^{*}P$$</p>
<p>and apply a modular inverse if necessary.</p>
<p>This is why the concept of NUMS is critical. The construction of a NUMS alternate generator is discussed in <a href="https://elementsproject.org/elements/confidentialtransactions/">the same CT doc as above</a>, and also in <a href="https://github.com/AdamISZ/ConfidentialTransactionsDoc/blob/master/essayonCT.pdf">my CT overview</a>, at the end of section 2.2. Note I use \(J\) here in place of \(H\) to avoid confusion with hash functions.</p>
<h3>Code and thoughts on implementation</h3>
<p>I did an abbreviated write up of the concept of this post in <a href="https://gist.github.com/AdamISZ/9cbba5e9408d23813ca8#defence2committingtoautxoinpublicplaintextatthestartofthehandshake">this gist</a>, as one of three possible ways of attacking the problem in Joinmarket: <a href="https://github.com/JoinMarketOrg/joinmarket/issues/156">how can we prevent people initiating transactions over and over again to collect information on utxos</a>? This algorithm is not intended as a <em>complete</em> solution to that issue, but it's very interesting in its own right and may have a variety of applications, perhaps.</p>
<p>The algorithm was fairly simple to code, at least in a naive way, and I did it some time ago using Bitcoin's <a href="https://github.com/bitcoincore/secp256k1">libsecp256k1</a> with the <a href="https://github.com/ludbb/secp256k1py">Python binding by ludbb</a>. An initial version of my Python "podle" module is <a href="https://github.com/JoinMarketOrg/joinmarket/blob/90ec05329e06beed0fbc09528ef6fb3d2c5d03ba/lib/bitcoin/podle.py">here</a>.</p>
<p>There are lots of tricky things to think about in implementing this; I think the most obvious issue is how would publishing/maintaining a public list work? If we just want each utxo to be allowed only one use, any kind of broadcast mechanism would be fine; other participants can know as soon as any \(H(P_2)\) is used, or at least to a reasonable approximation. Even in a multiparty protocol like Joinmarket, the utxo would be broadcast as "used" only after its first usage by each party, so it would from then on be on what is effectively a blacklist. But if the policy were more like "only allow reuse 3 times" this doesn't seem to work without some kind of unrealistic honesty assumption.</p>The matasano challenges20160622T19:03:52+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/thematasanochallenges/<p>On the <a href="http://cryptopals.com">cryptopals website</a> there is a list of cryptography coding challenges; originally they were hosted by matasano, so the name stuck (at least in my brain).</p>
<p>They're a kind of crash course in the basics of crypto in real world usage. And they're very nicely put together, as many can attest. I for one learnt a lot from them. You can see <a href="https://github.com/search?q=matasano">how popular they are on Github</a> (and you can also see how many people didn't get anywhere near finishing them!).</p>
<p>I did (most of it) in February this year; at that time only the first 6 sets were up, and so I did those. The 7th has recently been added, and the 8th more recently (though not on the site). You can see my solution set <a href="https://github.com/AdamISZ/matasanosolutions">here</a>. There was no attempt to organize it prettily; just one Python module per challenge.</p>
<p>If you are somehow inspired to try them yourself, you might find, as I did with other github repos, that others' solutions give a useful sanitycheck after you've finished each one.</p>
<h3>What did I learn?</h3>
<p>A lot of the issues that the challenges illustrate (although not all!) are kind of well known. Example: repeated nonces can be bad. But actually implementing attacks gives you a much richer understanding. Here are some things which I kind of sort of knew, but didn't really, until after these exercises.</p>
<ul>
<li><a href="https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Electronic_Codebook_.28ECB.29">ECB</a> isn't just bad because of penguins! It is not really a block cipher mode, and using it as such, you <em>will</em> have a bad time.</li>
<li><a href="https://en.wikipedia.org/wiki/Hamming_distance">Hamming distance</a> is a very cool idea. It's the key to the <a href="http://cryptopals.com/sets/1/challenges/6">first relatively hard challenge</a>.</li>
<li>Use nonces correctly in things like CTR, or cry (OK, this one is obvious, plus I already said it...)</li>
<li><a href="https://en.wikipedia.org/wiki/Padding_oracle_attack">Padding oracles</a> are a really, really nasty thing. The most sophisticated attacks in the sets are based around this idea.They can be bad (very) bad in CBC (the most famous TLS attacks were based around this: Beast, Lucky13 etc. all started from the original idea of <a href="http://www.iacr.org/cryptodb/archive/2002/EUROCRYPT/2850/2850.pdf">Vaudenay</a>), and Bleichenbacher's attacks of the same type are equally disastrous.</li>
<li>The widely used <a href="https://en.wikipedia.org/wiki/Mersenne_Twister">Mersenne twister</a> random number generator is profoundly cryptographically unsafe (duh?)</li>
<li>Daniel Bleichenbacher is your God</li>
</ul>
<p></p>
<h3>What is hard?</h3>
<p>Not everyone's the same, but I think a few are more likely than not to slow you down.</p>
<ul>
<li>the challenge on <a href="http://cryptopals.com/sets/3/challenges/23">recovering a Mersenne Twister seed</a> is ... well, twisty :)</li>
<li>And the padding oracle attacks, <a href="http://cryptopals.com/sets/3/challenges/17">on CBC</a> and <em>especially </em><a href="http://cryptopals.com/sets/6/challenges/48">on RSA with PKCS#1 1.5</a>, are really quite demanding. But, all the more satisfying for it :)</li>
<li>I actually hit a bit of a roadblock with <a href="http://cryptopals.com/sets/6/challenges/42">Bleichenbacher's e=3 RSA attack</a> too; as you can see from the comment <a href="https://github.com/AdamISZ/matasanosolutions/blob/master/challenges/matasano42.py#L115L118">here</a> I couldn't make Hal Finney's writeup work, and I'd be curious if I was wrong about the deduction I made there.</li>
</ul>
<p>Doubtless others too, that I forgot.</p>
<p></p>
<h3>Coding</h3>
<p>I read in a few places that people recommend using the challenges as a way to learn a new programming language. Hmm! What I'd say is that this kind of work is exactly where Python shines, as it's very good for quickanddirty (rather than architecturally thoughtful) algorithm implementation requiring a lot of data of different types. I hardly ever had to think about code while doing it, which helped a lot. As for dependencies, I managed to get by only with an AES implementation (slowaes) and Crypto.RSA (for generating primes; you implement RSA yourself). Of course, this hardly counts as a disrecommendation of other languages :)</p>
<p></p>
<h3>The most profound lesson of all...</h3>
<blockquote>
<p><em>I'm back and I'm ringin' the bell </em><br/><em>A rockin' on the mike while the fly girls yell </em><br/><em>In ecstasy in the back of me </em><br/><em>Well that's my DJ Deshay cuttin' all them Z's </em><br/><em>Hittin' hard and the girlies goin' crazy </em><br/><em>Vanilla's on the mike, man I'm not lazy.</em></p>
</blockquote>
<p>Only matasano vets would understand :)</p>About me20160622T16:16:43+00:00Adam Gibsonhttps://joinmarket.me:8001/blog/blog/author/adam/https://joinmarket.me:8001/blog/blog/aboutme/<h3>Just some basic contact info in this post.</h3>
<p>Adam Gibson.</p>
<p>Github: <a href="https://github.com/AdamISZ/">page</a></p>
<p>Reddit, bitcointalk, freenode: username <strong>waxwing</strong>, twitter: username @waxwing__ (no longer used), mastodon <a href="https://mastodon.social/@waxwing">waxwing</a>.</p>
<p>My gpg public key can be found <a href="http://keys.gnupg.net/pks/lookup?op=vindex&fingerprint=on&search=0x141001A1AF77F20B">here</a> notice this new one is signed with the old one which is expiring Dec 2018, the fingerprint is attached to some of the above accounts too, and also can be found in the pubkeys/ directory of the <a href="https://github.com/JoinmarketOrg/joinmarketclientserver">Joinmarket repo</a></p>
<p></p>
<p>I have bits and pieces of experience in software (nowadays mostly Python), nuclear engineering, finance, physics and mathematics (and the teaching thereof).</p>
<p></p>
<p>If you want to contact me via email, my username is ekaggata for the most popular public email service.</p>
<p></p>
<p>Living in Daugavpils, southern Latvia, originally from London, UK.</p>
<p></p>
<p></p>
<p></p>