Storage query.multi subscription starts to provide 0 values after short period of time.

[jak-pan]

@nohaapav has been investigating an issue where we start to get 0 values after a couple of blocks to most of the query.multi subscription responses.

We have been able to track this issue to version 14.1.1 which included #5997 that I suspect could be the culprit. All of the previous versions work as expected and all of the later versions including this one are showing the same behaviour.

Sample snippet included. The main part is the multi query subscription here. This is a part of larger codebase but I think it's good as an example.

https://gist.github.com/jak-pan/91d5e9cba4550c73c6fdc0708e61e427

Sample response

https://gist.github.com/jak-pan/9699ceb399099779216edfc12c17756d

You can see that after a few blocks most of the values got wiped. This is not the case with 14.0.1 and earlier versions. Setting the cacheCapacity to 0 doesn't help.

[TarikGul]

Thank you for the reproduction and example case, adding this to the High priority so it's prioritized and looked into!

[TarikGul]

cc: @filvecchiato

[TarikGul]

Pulling influence from your script - I wrote something a little more complete. I wasn't able to reproduce, I will try and tweek a bit more.

First script that does a balance transfer every 8 seconds:

import { promises as fs } from "fs";
import { ApiPromise, WsProvider } from '@polkadot/api'
import { Keyring } from '@polkadot/keyring';

const createKeyPair = async () => {
  const keyPhrase = await fs.readFile('key.txt', 'utf-8')
      .catch((err) => {
          console.error(err);
          process.exit(1);
      });
  const keyring = new Keyring({ type: 'sr25519', ss58Format: 42 });

  return keyring.addFromUri(keyPhrase, { name: 'keyPair' });
}

const accounts = [
    '5Fv5R9yFakshShuexq8VmzUzhT2Aobuh93p48YKEj7VPf54s',
    '5F4vtwdsBu2ezqNP887qzPzDkp8D6XAQFRf74GdbsokMpp7A',
    '5HdVyXtYLjrcdsqaSK6g7jwfVhbiwG8fcWeWwCsqegzwxjvb',
    '5ECy5bSdJqHuV9U5doPpjv7Fw8snjrEVsGm2aKPQiZagZU1K',
    '5HdVyXtYLjrcdsqaSK6g7jwfVhbiwG8fcWeWwCsqegzwxjvb'
]

const main = async () => {
    const api = await ApiPromise.create({
        provider: new WsProvider('wss://westend-rpc.polkadot.io')
    });

    await api.isReady;

    const keypair = await createKeyPair();

    let i = accounts.length;
    setInterval(async () => {
        i++
        try {
            const res1 = await api.tx.balances.transferAllowDeath(accounts[i % accounts.length], 100000000000).signAndSend(keypair, { tip: 1000000 });
            console.log(`Transaction result: ${res1}`);
        } catch {
            // Failed: There was an error - do nothing
        }
    }, 8000)
}

main();

Subscription that does the multi call:

import { ApiPromise, WsProvider } from '@polkadot/api'

const main = async () => {
  const api = await ApiPromise.create({
    provider: new WsProvider('wss://westend-rpc.polkadot.io')
  });

    const accounts = [
        '5Fv5R9yFakshShuexq8VmzUzhT2Aobuh93p48YKEj7VPf54s',
        '5F4vtwdsBu2ezqNP887qzPzDkp8D6XAQFRf74GdbsokMpp7A',
        '5HdVyXtYLjrcdsqaSK6g7jwfVhbiwG8fcWeWwCsqegzwxjvb',
        '5ECy5bSdJqHuV9U5doPpjv7Fw8snjrEVsGm2aKPQiZagZU1K',
        '5HdVyXtYLjrcdsqaSK6g7jwfVhbiwG8fcWeWwCsqegzwxjvb'
    ]

  const unsub = api.query.system.account.multi(accounts, (balances) => {
    console.log('===========', new Date().toLocaleString(), '===========');

    balances.forEach(b => {
      console.log(b.data.free.toString())
    })
  });

}

main()

[jak-pan]

I'll try to get you complete example.

[nohaapav]

Try api.query.tokens.accounts.multi instead of system .. System pallet was working just fine i guess.