Skip to content

Update documentation using deterministic generation#8557

Open
rustyrussell wants to merge 15 commits into
ElementsProject:masterfrom
rustyrussell:guilt/fix-doc-gen
Open

Update documentation using deterministic generation#8557
rustyrussell wants to merge 15 commits into
ElementsProject:masterfrom
rustyrussell:guilt/fix-doc-gen

Conversation

@rustyrussell

@rustyrussell rustyrussell commented Sep 19, 2025

Copy link
Copy Markdown
Contributor

Depends on #8556 Merged!

Now we do major surgery on autogenerate-rpc-examples.py. Finally we run it to produce the canned blocks and wallet for future runs, and re-enable it in CI.

@rustyrussell rustyrussell added this to the v25.12 milestone Sep 19, 2025

@ShahanaFarooqui ShahanaFarooqui left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rustyrussell The changes in ./tests/autogenerate-rpc-examples.py look good at first glance, but I am unable to test them locally due to missing test data files:

./tests/data/autogenerate-bitcoin-blocks.json
./tests/data/autogenerate-bitcoind-wallet.dat

Could you please add these files or provide instructions on how to generate them for local testing?

@ShahanaFarooqui

Copy link
Copy Markdown
Collaborator

Now uv run make update-doc-examples is throwing error:

15:12:18 - ERROR - Error in setting up nodes: txid f160d78f39a5ccb96c1439333787b98f1a37c1bb3765b34c74047c8895f4e397 not found
15:12:18 - ERROR - txid f160d78f39a5ccb96c1439333787b98f1a37c1bb3765b34c74047c8895f4e397 not found
Traceback (most recent call last):
  File "/home/shahana/workspace/lightning/tests/autogenerate-rpc-examples.py", line 1443, in test_generate_examples
    l1, l2, l3, l4, l5, l6, c12, c23, c25 = setup_test_nodes(node_factory, bitcoind, regenerate_blockchain)
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/shahana/workspace/lightning/tests/autogenerate-rpc-examples.py", line 331, in setup_test_nodes
    c12, c12res = l1.fundchannel(l2, FUND_CHANNEL_AMOUNT_SAT)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/shahana/workspace/lightning/contrib/pyln-testing/pyln/testing/utils.py", line 1194, in fundchannel
    assert False, f"txid {res['txid']} not found"
           ^^^^^
AssertionError: txid f160d78f39a5ccb96c1439333787b98f1a37c1bb3765b34c74047c8895f4e397 not found

And uv run make update-doc-examples-newchain is throwing:

15:10:44 - ERROR - Error in generating transactions examples: RPC call failed: method: fetchbip353, payload: {'address': 'send.some@satsto.me'}, error: {'code': -32700, 'data': None, 'message': 'failed to fetch payment instructions: HrnResolutionError("Multiple TXT records existed for the HRN, which is invalid")'}
15:10:44 - ERROR - RPC call failed: method: fetchbip353, payload: {'address': 'send.some@satsto.me'}, error: {'code': -32700, 'data': None, 'message': 'failed to fetch payment instructions: HrnResolutionError("Multiple TXT records existed for the HRN, which is invalid")'}
Traceback (most recent call last):
  File "/home/shahana/workspace/lightning/tests/autogenerate-rpc-examples.py", line 1444, in test_generate_examples
    c23_2, c23res2, c34_2, inv_l11, inv_l21, inv_l22, inv_l31, inv_l32, inv_l34 = generate_transactions_examples(l1, l2, l3, l4, l5, c25, bitcoind)
                                                                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/shahana/workspace/lightning/tests/autogenerate-rpc-examples.py", line 463, in generate_transactions_examples
    update_example(node=l1, method='fetchbip353', params={'address': 'send.some@satsto.me'}, description=['Example of fetching BIP-353 payment details.'])
  File "/home/shahana/workspace/lightning/tests/autogenerate-rpc-examples.py", line 260, in update_example
    response = node.rpc.call(method, params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/shahana/workspace/lightning/contrib/pyln-testing/pyln/testing/utils.py", line 811, in call
    res = LightningRpc.call(self, method, payload, cmdprefix, filter)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/shahana/workspace/lightning/contrib/pyln-client/pyln/client/lightning.py", line 428, in call
    raise RpcError(method, payload, resp['error'])
pyln.client.lightning.RpcError: RPC call failed: method: fetchbip353, payload: {'address': 'send.some@satsto.me'}, error: {'code': -32700, 'data': None, 'message': 'failed to fetch payment instructions: HrnResolutionError("Multiple TXT records existed for the HRN, which is invalid")'}

@rustyrussell rustyrussell modified the milestones: v25.12, v26.03 Nov 13, 2025
@sangbida sangbida modified the milestones: v26.04, 26.06 Mar 18, 2026
@madelinevibes madelinevibes modified the milestones: v26.06, v26.09 May 6, 2026
@madelinevibes madelinevibes added the Status::Ready for Review The work has been completed and is now awaiting evaluation or approval. label Jun 17, 2026
…w one via splice.

This happens if the channel is *not* announcable yet.  Then we hit the assertion
in funding_depth_cb that the txid is the same as the current funding.txid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: fixed crash when we splice a channel which hasn't been announced yet.
…LN_DEV_ENTROPY_SEED

Only in developer mode, ofc.

Notes:
1. We have to move the initialization before the lightningd main trace_start,
   since that uses pseudorand().
2. To make the results stable, we need to use per-caller values to randbytes().
   Otherwise external timing changes the call order.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This allows us to override it for deterministic results.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We temporarily add a flake8 exception so it doesn't complain about all
the now-unused variables.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
…hannel example.

Now order has changed, the first one is insufficient; this is more robust.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Unless REGENERATE_BLOCKCHAIN is true, in which case we make them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This makes our transaction generation (nlocktime) consistent, as well as our htlcs
for payment.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This makes many of our information outputs consistent.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
1. Don't mangle bookkeeper events.
2. Use l3, not l1, for listtransactions: it's more stable.
3. Use l2, not l1, for listpeerchannels and listchannels.
3. Use l2, not l2, for listfunds.

(l1 has a lot of variation).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
And hand in CLN_NEXT_VERSION to replace the version string.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is `make update-doc-examples-newchain`, which creates the canned blocks (and,
since it's the first time, the bitcoind wallet).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Status::Ready for Review The work has been completed and is now awaiting evaluation or approval.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants