What happens to "unselected" transactions?

[Andrey]

One thing has been puzzling me.

In reading several topics in this board, I understand that solo miners (if they still exist) and pools have the option to select the transactions that will be included in their blocks.

So if a pool in creating a new block selects e.g., 6 transactions #1, #2, #4, #5, #6 and #7, deciding to leave out #3 which e.g., had a lower fee than other transactions.

They calculate the merkle root(s), create the block header(s), etc. and pool members start hashing their respective headers.

Someone in the pool uses a nonce that causes the hash to be lower than the target, and that block is added to the blockchain.

My question simply is, how does transaction #3 ever get to the blockchain?

 

[Marvin]

There's a new block every 10 minutes. If the network is saturated with high fee transactions, the transaction will take a space where #3 is viable to add then it will be added, after about 2 weeks transactions can be dropped from the network although may or may not be depending on the node.

If a low fee transaction is sent, you can cpfp with it if you received it (so you spend the funds immediately, even to the same address but with a higher fee to compensate) or you can rbf if your wallet is compatible with it where you can double spend your transaction with a higher fee (miners favour higher fee txs so they will drop the low fee one)

 

[Mathew]

It gets picked up in some subsequent block or it doesn't. Unless/until one of its inputs is spend some other way the transaction remains valid and can be included ones it makes sense to include it.


I'm not sure how approximate you intended your example to be. The actual behaviour is that miners collect up all translations they receive (above some very low threshold minimum fee to prevent DOS attacks) and queue them. Then they try to construct blocks that make the most fee they can, leaving out whatever doesn't fit-- which get included later when there are fewer higher paying things available.

E.g. This graph from a couple years ago shows the fees available for a block immediately before a new block was found (red), and the fees available in a block created instantly after a block was found (green). So all those fees in green were already available but wouldn't fit.

As the subsidy goes away having a backlog is important to keeping hashrate up ... otherwise miners would have to turn off and stop mining as soon as a block is found and wait for there to be enough transactions to keep mining.

 

[Jamari]

Thanks for the explanation thus far.

Maybe if I understood the mechanism that allows transactions to be available for selection, this would make more sense.

The idea that transaction #3 (or whatever) under certain circumstances could take hours to get to the blockchain (or never get there if hanging out to dry for 2 weeks) seems "inefficient".

Could someone explain where these transactions are before they get into a block, and how a pool gains access to them to select (or not).

Thanks.

 

[Mauricio]

Users with a full node can broadcast the transaction and the owner can actually choose to re-broadcast it with higher fees. Although that requires some more technical steps.

However, the simplest solution is to just wait as a miner would eventually pick it up form the mempool. Although with some delay probably.

 

[Niko]

Could someone explain where these transactions are before they get into a block, and how a pool gains access to them to select (or not).

The node where it was broadcast will save it in its mempool then relay that particular TX to its peers and those peers will also save and relay it as long as it complies with their relay/default settings;
after a short while, it will eventually reach a miner/pool's node.

For the meantime if it didn't get mined, it'll stay in the node's "mempool" as long as it wasn't "mined",
eg. saved in the data directory of their bitcoin core as mempool.dat.

 

[Isaias]

Everything posted has been interesting; indeed learning is generally interesting, but enough philosophy...

Both alani123 and nc50lc mentioned the mempool - a clue.

At the advise of some random post found by searching mempool.dat, I went to my full node wallet (which continues to just eat up disk space but it's what I started with so...) and typed:

Code:
bitcoin-cli getrawmempool true

and before me appeared data, which led to a better understanding and more questions:

The file was 9MB in size and dated 2 days ago so I wasn't sure if that meant that I didn't have the last 2 days of transactions or, as I've seen before, the OS doesn't change the date even through some file has been updated

9MB of transactions that weren't picked up? Seems like a lot. (in JSON format, each transaction took 767 bytes so = approx 11,700 transactions)

I then noted 2 fields in particular:

Code:
"time": 1573801222,
"height": 603870,

If this is a transaction that hasn't seen a block yet, why does it have a height.

I didn't know how to translate the date field - help appreciated.

So the questions are:

Does the mempool only contain "unselected" transactions?

If so, why does it have a block height?

What does this date field represent? I'm sure there's an easy way to translate it e.g., some online tool - if anyone knows of one, please let me know.

If this does contain transactions already in the blockchain, which field indicates that it was never selected (or never made its way to the blockchain)?