What Libbitcoin and SX are And Why They Matter

Amir Taaki’s libbitcoin has come to be one of the most advanced alternative implementations of the Bitcoin protocol in existence. Although there are now dozens of alternative implementations on the market, libbitcoin is one of the few that re-implements the complete Bitcoin standard, allowing users to run a deterministic wallet, an elliptic curve message signing interface and, of course, a fully functional Bitcoin node. Even multisignature transactions, which many popular wallets including blockchain.info and Bitcoin Wallet for Android still do not support, libbitcoin handles just fine. Libbitcoin is also unique in its modularity; the package itself is a software library, containing a set of components each of which can be programmatically called individually, avoiding the need to start up a fully fledged, monolithic software daemon just to make a few simple transactions or queries. If you want a full client running locally, you can use libbitcoin to do that, but you can also just as easily only use libbitcoin for transaction management and networking and rely on other nodes for transaction data. Whatever you need, you use, and whatever you don’t need you simply don’t bother with.

As the first major practical application of libbitcoin, Amir Taaki has recently released sx, a set of command line utilities for working with Bitcoin keys and transactions. The basic intent is simple: empower Bitcoin users who are somewhat technically skilled, but not experienced programmers, to interact with Bitcoin not just as an interface where they type in an amount and a destination address and bitcoind does everything for them, but also work directly with the underlying building blocks. Every sx tool follows the Unix philosophy: it does one thing, and it does it well.

genaddr

takes an ELECTRUM wallet seed or master public key and an index and returns an address.

mktx

creates unsigned transactions, and

sign-input

signs transaction inputs.

history

returns a list of unspent transactions associated with a Bitcoin address that can be used as inputs – and so on.

These tools can be accessed on the command line to construct Bitcoin transactions one step at a time, or one can call them programatically (eg. with

os.popen(command).read()

in Python) to create Bitcoin software like merchant packages and wallets. Up until now, creating a new Bitcoin wallet was difficult – one would need to either re-implement everything from scratch, take an existing wallet and work hard to untangle the code to separate out the transaction logic from the wallet logic, or rely on a bitcoind node running in the background. With libbitcoin and sx, however, the tools to work with Bitcoin transactions on the lowest level are already there – freeing developers to work on the problems that truly need the most work – like optimizing security and the user experience.

More Than Just A Library

So why do libbitcoin and SX matter? As it turns out, the answer goes far beyond the simple fact that they’re convenient. One of the main problems with the Bitcoin ecosystem is that although Bitcoin is nominally decentralized, in reality the miners that keep the network running are all running the same piece of core Bitcoin software: bitcoind. Bitcoind is the “headless” (ie. command line only) version of the original Bitcoin client that has been developed ever since Satoshi released the first version in 2009, and serves as the software that all miners and nearly all businesses use to communicate with the Bitcoin network.

This is unhealthy for two reasons. First, having a monopoly can potentially lead to highly disruptive blockchain forks; for example, in March this year the removal of a database bug preventing bitcoind nodes from processing a block simultaneously affecting the status of more than 5,000 transactions caused the Bitcoin network to split in half for about six hours as nodes with the bug and without the bug disagreed on the validity of a block and started working on two separate blockchains. The fix: the Bitcoin community came together and agreed to shut down the version 0.8 nodes, and added the database bug to the official protocol for two months until it could be removed in a controlled way two months later. If there were instead five different Bitcoin implementations in active and widespread use, the failure of any one would only affect a small portion of the Bitcoin network – and ordinary users’ clients, which typically only verify transactions and not blocks, would probably not be affected at all as, in the event of a fork, the “legitimate” blockchain would nearly always be the one commanding a majority of the network.

The second reason is political. Many core bitcoind developers, particularly lead developer Gavin Andresen, are also part of the Bitcoin Foundation, an organization which has taken clear steps toward using political lobbying as a strategy for protecting Bitcoin. The fear is that, in the future, the government might try to regulate Bitcoin more, and give the foundation an ultimatum: you include certain privacy-eroding features (a moderate example might be an “address validation fee” to discourage people from covering their tracks by creating many Bitcoin addresses) in the protocol, or we ban Bitcoin exchanges. The Foundation may well comply, and the US government knows it. If the Foundation did not have the power to make these kinds of changes to the protocol without widespread international consent, however, then the US government would not have this option – they would be forced to either try banning Bitcoin outright, at the cost of massive goodwill among the tech and open-source community at the very least, or let it be and focus on softer regulatory strategies. This is why Bitcoin advocates like Amir Taaki, Joerg Platzer and others have long supported diversifying and internationalizing the Bitcoin ecosystem. And libbitcoin just might be the catalyst that finally sets the process going.

How do you use SX?

Installing sx is very simple if you’re on Linux; on the command line,

wget http://sx.dyne.org/install-sx.sh

followed by

sudo bash ./install-sx.sh

is all it takes. If you’re on Windows or Mac, things are somewhat more complicated, although not more so than any other similar software package; the source code is available for download at https://github.com/genjix/sx, and from there you can install it manually.

Once sx is installed, here are some of the things you can do with it. First, the basic address and key manipulations.

Generate a private key/public key/address triple:

sx newkey > pk1cat pk1 | sx pubkey > pub1cat pk1 | sx addr > addr1cat pk1 pub1 addr1

Generate private key/address pairs from an Electrum wallet:

sx newseed > seedfor x in `seq 0 9`; do cat seed | sx genpriv $x; cat seed | sx genaddr $x; done

Generate just the addresses from a master public key:

cat seed | sx mpk > mpkfor x in `seq 0 9`; do cat mpk | sx genaddr $x; done

Transactions

Now, it’s time to get into transactions. First, a little understanding is needed as to how transactions work. Bitcoin does not have the concept of “accounts”; instead, a transaction spending money from an address must directly reference some transactions sending at least an equal amount of money to that address; these are usually called “previous transaction outputs”. For example, if you have an address to which you sent 2 BTC, 3 BTC and then 4 BTC and want to spend 5 BTC, you are not spending 5 BTC out of that address; rather, you are spending the 2 BTC and 3 BTC transaction outputs. Additionally, an output cannot be “partially spent”; even if you only want to send 0.1 BTC, you must consume an entire output. You can avoid wasting 1.9 BTC by adding a special output to the transaction, called “change”, where you send the remaining funds back to yourself.

The basic command that you will use to get these unspent transaction outputs is

sx history

:

sx history `cat addr1`

However, this by itself returns all transactions connected to that address, not just the ones we want, so we need to filter the output a bit. This is a complicated script; basically, it first looks for the word “Unspent” in the history and gets those lines plus the transaction data in the two lines above them. Then, it uses grep and awk to extract just the data that we need. Note that the command returns values in satoshis; 100 million satoshis equal 1 BTC. Send 0.001 BTC to the address contained in the addr1 file (

cat addr1

to see it, and

sx qrcode `cat addr1` qrcode.png

to generate a scannable QR code), and run the following command:

sx history `cat addr1` | grep Unspent -B 2 | grep output | awk '{print $2}' > inputcat input

Now that we have the history, let’s use sx to create and send a transaction.

sx mktx txfile.tx `cat input` -o 18qk7SqRHuS4Kf3f6dmsvqqv7iw1xy77Z6:90000sx rawscript dup hash160 [ `cat addr1 | sx decode-addr` ] equalverify checksig > raw.scriptcat pk1 | sx sign-input txfile.tx 0 `cat raw.script` > sigsx rawscript [ `cat sig` ] [ `cat pub1` ] | sx set-input txfile.tx 0 > txfile2.tx; mv txfile2.tx txfile.txsx broadcast-tx txfile.tx

Here’s what to do if you have multiple transaction inputs you want to spend. Send 0.0002 BTC to your address five times, and run the above sx history command again and make sure that it returns five outputs. To avoid having to copy and paste five transaction inputs by hand, we’ll do a little more command line magic:

sx history `cat addr1` | grep Unspent -B 2 | grep output | awk '{print $2}' > temp1cat temp1 | sed 's/^/-i /' | tr 'n' ' ' > temp2cat temp2

Notice how temp2 contains all the inputs in exactly the right format for

sx mktx

. Now, we just splice them in:

sx mktx txfile.tx `cat temp2` -o 18qk7SqRHuS4Kf3f6dmsvqqv7iw1xy77Z6:90000sx rawscript dup hash160 [ `cat addr1 | sx decode-addr` ] equalverify checksig > raw.scriptfor x in `seq 0 4`; do cat pk1 | sx sign-input txfile.tx $x `cat raw.script` > sig$x; donefor x in `seq 0 4`; do sx rawscript [ `cat sig$x` ] [ `cat pub1` ] | sx set-input txfile.tx $x > txfile2.tx; mv txfile2.tx txfile.tx; donesx broadcast-tx txfile.tx,

The Holy Grail: Multisignature Transactions

First we’ll generate 3 sets of private keys, pubkeys and addresses.

for x in `seq 1 3`; do sx newkey > pk$x; donefor x in `seq 1 3`; do cat pk$x | sx pubkey > pub$x; donefor x in `seq 1 3`; do cat pk$x | sx addr > addr$x; done

Then, we’ll create the multisig address.

sx rawscript 2 [ `cat pub1` ] [ `cat pub2` ] [ `cat pub3` ] 3 checkmultisig > msig.scriptcat msig.script | sx showscriptcat msig.script | sx scripthash > 3addrcat 3addr

Send 0.001 BTC to the address (doesn’t work with all wallets!) and then run the following to get the transaction hash:

sx history `cat 3addr` | grep Unspent -B 2 | grep output | awk '{print $2}' > inputcat input

Now, we need to construct the transaction and sign it, in this case using the first and third private keys (first and second or second and third work equally well). We’ll send only 50000 satoshis, leaving 0.0005 BTC as a fee.

sx mktx txfile.tx -i `cat input` -o 18qk7SqRHuS4Kf3f6dmsvqqv7iw1xy77Z6:50000cat pk1 | sx sign-input txfile.tx 0 `cat msig.script` > sig1cat pk3 | sx sign-input txfile.tx 0 `cat msig.script` > sig3sx rawscript zero [ `cat sig1` ] [ `cat sig3` ] [ `cat msig.script` ] > input.scriptcat input.script | sx set-input txfile.tx 0

And, finally, broadcast the transaction.

sx broadcast-tx txfile.tx

Fairly complicated, but multisignature transactions are not exactly simple in any case. With sx, however, the complexity is reduced to a series of steps that you can simply follow, or even incorporate into your own programs to run whenever you need to. Whether you’re a command line enthusiast, a Bitcoin developer or just someone interested in looking more deeply into how Bitcoin transactions work, sx is the tool for you. Happy hacking!