Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: update fulcrum and adapt config to large wallets #7322

Merged
merged 1 commit into from
Dec 10, 2024

Conversation

openoms
Copy link
Contributor

@openoms openoms commented Dec 9, 2024

Config details from: https://github.com/cculianu/Fulcrum/blob/master/doc/fulcrum-example-config.conf

max_subs_per_ip = 1000000 does allow to sync wallets tracking more than the default 75000 addresses.

The global max_subs limit (DEFAULT: 10M) still applies.

# Maximum subscriptions (global limit) - 'max_subs' - DEFAULT: 10000000
#
# The maximum number of simultaneous script hash subscription that the server
# supports. If more than this number of subscriptions is requested by clients
# (unlikely) then the server will refuse new subscription requests. It will then
# kick off a background task to find the most-subscribed IP address and kick all
# the clients coming from that IP, and then clean up the subscription table.
#
# This config parameter was added in Fulcrum 1.0.5 as a measure to prevent
# clients from potentially exhausting server resources.
#
#max_subs = 10000000


# Maximum subscriptions (per-IP address) - 'max_subs_per_ip' - DEFAULT: 75000
#
# The maximum number of script hash subscriptions per IP address. Note that IP
# addresses matching 'subnets_to_exclude_from_per_ip_limits' will not be
# subjected to this limit (they will still be subjected to the global 'max_subs'
# limit, however).
#
# If a client attempts to subscribe beyond this limit for all the connections
# coming from their IP address, it will receive an error message.
#
#max_subs_per_ip = 75000

utxo_cache was called fast-sync previously.
It only affects the initial sync of fulcrum:

#-------------------------------------------------------------------------------
# OPTIONS USED ONLY DURING INITIAL SYNC
#-------------------------------------------------------------------------------

# UTXO cache = 'utxo_cache' - DEFAULT: 0
#
# If specified, Fulcrum will use a UTXO Cache that consumes extra memory but
# syncs up to to 2X faster. To use this feature, you must specify a memory value
# in MB to allocate to the cache. It is recommended that you give this facility
# at least 500 MB for it to really pay off, although any amount of memory given
# (minimum 64 MB) should be beneficial. Note that this feature is currently
# experimental and the tradeoffs are: it is faster because it avoids redundant
# disk I/O, however, this comes at the price of considerable memory consumption
# as well as a sync that is less resilient to crashes mid-sync. If the process
# is killed mid-sync, the database may become corrupt and lose UTXO data. Use
# this feature only if you are 100% sure that won't happen during a sync.
# The default is off (0). This option only takes effect on initial sync,
# otherwise this option has no effect.
#
#utxo_cache = 0

@openoms openoms requested a review from dolcalmi December 9, 2024 13:03
@openoms
Copy link
Contributor Author

openoms commented Dec 9, 2024

Tested locally with uncommenting the part:
https://github.com/GaloyMoney/charts/blob/main/dev/bitcoin/Tiltfile#L74-L82

Runs ok on regtest:

[2024-12-09 12:24:47.004] Fulcrum 1.11.1 (Release a1f6a54) - Mon Dec 9, 2024 12:24:47.004 UTC - starting up ...
[2024-12-09 12:24:47.005] Max open files: 1048576
[2024-12-09 12:24:47.008] Loading database ...
[event: pod galoy-dev-bitcoin/fulcrum-0] Startup probe failed: dial tcp 10.42.0.111:8080: connect: connection refused
[2024-12-09 12:24:49.451] DB [redacted secret galoy-hydra:dsn]: 512.00 MiB
[2024-12-09 12:24:49.451] Verifying headers ...
[2024-12-09 12:24:49.451] DB version: v3
[2024-12-09 12:24:49.451] BitcoinDMgr: starting 1 bitcoin RPC client ...
[2024-12-09 12:24:49.452] BitcoinDMgr: started ok
[2024-12-09 12:24:49.465] Stats HTTP: starting 1 server ...
[2024-12-09 12:24:49.465] Starting listener service for HttpSrv 0.0.0.0:8080 ...
[2024-12-09 12:24:49.564] Service started, listening for connections on 0.0.0.0:8080
[2024-12-09 12:24:49.776] <BitcoinDMgr> Coin: BTC
[2024-12-09 12:24:49.899] <Controller> Chain: [redacted secret network:network]
[2024-12-09 12:24:49.899] <Controller> Block height 1, downloading new blocks ...
[2024-12-09 12:24:49.899] <Controller> utxo-cache: Enabled; UTXO cache size set to 1024000000 bytes (available physical RAM: 9984454656 bytes)
[2024-12-09 12:24:50.159] <Controller> Processed 2 new blocks with 2 txs (2 inputs, 3 outputs, 2 addresses), verified ok.
[2024-12-09 12:24:50.160] <Controller> Initial sync ended, flushing and deleting UTXO Cache ...
[2024-12-09 12:24:50.176] <Controller> Storage UTXO Cache: Flushing to DB ...
[2024-12-09 12:24:50.194] <Controller> Initializing header merkle cache ...
[2024-12-09 12:24:50.194] <Controller> Block height 1, up-to-date
[2024-12-09 12:24:50.194] SrvMgr: starting 4 services ...
[2024-12-09 12:24:50.194] Starting listener service for TcpSrv 0.0.0.0:50001 ...
[2024-12-09 12:24:50.207] Service started, listening for connections on 0.0.0.0:50001
[2024-12-09 12:24:50.223] Starting listener service for SslSrv 0.0.0.0:50002 ...
[2024-12-09 12:24:50.232] Service started, listening for connections on 0.0.0.0:50002
[2024-12-09 12:24:50.233] Starting listener service for WsSrv 0.0.0.0:50003 ...
[2024-12-09 12:24:50.233] Service started, listening for connections on 0.0.0.0:50003
[2024-12-09 12:24:50.233] Starting listener service for WssSrv 0.0.0.0:50004 ...
[2024-12-09 12:24:50.233] Service started, listening for connections on 0.0.0.0:50004
     ┊ Scheduled       - 1s
     ┊ Initialized     - 1m58s
     ┊ Ready           - 24s

@openoms openoms merged commit c3805b2 into main Dec 10, 2024
3 of 4 checks passed
@openoms openoms deleted the fulcrum-update-and-config branch December 10, 2024 06:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants