3

Whisper Protocol is not well documented related to its usage. I believe this issue will help other users who might face with similar problem.

Each sent message has time-to-live in seconds (ttl), that is defined by the sender. For more detail please see Whisper v5 RPC API => shh_post. Whisper Protocol keeps received messages in the memory until messages' time-to-live time is over, when geth was closed or shh.newMessageFilter() is not activated.

=> How could we access and print those received messages that are actually stored in the memory?

Whisper v5 RPC API:

shh_getFilterMessages

Retrieve messages that match the filter criteria and are received between the last time this function was called and now.

  • But there is no getFilterMessages() method with the latest shh. Please see.

On this solution; we can use same key pairs after geth is restarted. But the issue after geth is restarted and later when we run shh.newMessageFilter(); it does not retrieve/return any message that was sent while geth was closed.

[Q] How could we obtain/filter messages that sent when geth was closed or shh.newMessageFilter() is not activated?. When we run web3.shh.newMessageFilter(), it only retrieves/filters messages right after it start running and it does not filter/print any message before it starts running.

Example Scenario:

=> On the receiver node I closed geth and from the sender node, I have sent multiple messages into receiver node. Later, when I reopened geth on the receiver node, shh.info.memory verifies that there are received messages in memory, but I was not able to retrieve/filter those messages using web3.shh.newMessageFilter(). 

> shh.info.memory
780

Here is some logs:

Creating web3.shh.newMessageFilter():

> kId=shh.addPrivateKey('0x8de0b39f3a4a109f27bb06a92dab680d58e8f029eaca61da6be7115981a2ed11')
"439bfd6aa63a81b404ba20e4fc250e11c326d72ad9943c72dc445f476638989d"

> filter=web3.shh.newMessageFilter(
...         {privateKeyID:kId},
...         function(err, res) {console.log(web3.toUtf8(res.payload))});
{
  callbacks: [function(err, res)],
  filterId: "a0b4194651c9143d977c2fa4848ab681b414720364461b4096fb17588d84ae78",
  formatter: null,
  getLogsCallbacks: [],
  implementation: {
    getLogs: function(),
    newFilter: function(),
    poll: function(),
    uninstallFilter: function()
  },
  options: {
    privateKeyID: "439bfd6aa63a81b404ba20e4fc250e11c326d72ad9943c72dc445f476638989d"
  },
  pollFilters: [],
  requestManager: {
    polls: {
      82f08d2d7fe974a69acdadcfac39f788575d337eed9577969b617ae28462e597: {
        data: {...},
        id: "82f08d2d7fe974a69acdadcfac39f788575d337eed9577969b617ae28462e597",
        callback: function(error, messages),
        uninstall: function()
      },
      a0b4194651c9143d977c2fa4848ab681b414720364461b4096fb17588d84ae78: {
        data: {...},
        id: "a0b4194651c9143d977c2fa4848ab681b414720364461b4096fb17588d84ae78",
        callback: function(error, messages),
        uninstall: function()
      }
    },
    provider: {
      newAccount: function(),
      openWallet: function(),
      send: function github.com/ethereum/go-ethereum/console.(*bridge).Send-fm(),
      sendAsync: function github.com/ethereum/go-ethereum/console.(*bridge).Send-fm(),
      sign: function(),
      unlockAccount: function()
    },
    timeout: {},
    poll: function(),
    reset: function(keepIsSyncing),
    send: function(data),
    sendAsync: function(data, callback),
    sendBatch: function(data, callback),
    setProvider: function(p),
    startPolling: function(data, pollId, callback, uninstall),
    stopPolling: function(pollId)
  },
  get: function(callback),
  stopWatching: function(callback),
  watch: function(callback)
}

I can see shh_getFilterMessages under filter.requestManager.polls but I cannot call it.

> filter.requestManager.polls
{
  82f08d2d7fe974a69acdadcfac39f788575d337eed9577969b617ae28462e597: {
    data: {
      method: "shh_getFilterMessages",
      params: ["82f08d2d7fe974a69acdadcfac39f788575d337eed9577969b617ae28462e597"]
    },
    id: "82f08d2d7fe974a69acdadcfac39f788575d337eed9577969b617ae28462e597",
    callback: function(error, messages),
    uninstall: function()
  },
  a0b4194651c9143d977c2fa4848ab681b414720364461b4096fb17588d84ae78: {
    data: {
      method: "shh_getFilterMessages",
      params: ["a0b4194651c9143d977c2fa4848ab681b414720364461b4096fb17588d84ae78"]
    },
    id: "a0b4194651c9143d977c2fa4848ab681b414720364461b4096fb17588d84ae78",
    callback: function(error, messages),
    uninstall: function()
  }
}

> a.requestManager.polls.82f08d2d7fe974a69acdadcfac39f788575d337eed9577969b617ae28462e597
(anonymous): Line 1:23 Unexpected token ILLEGAL

web3.py approach:

from web3 import Web3, HTTPProvider
web3 = Web3(HTTPProvider('http://localhost:8545'))
from web3.shh import Shh
Shh.attach(web3, "shh")

import time
print(web3.shh.info)

keyPair = web3.shh.newKeyPair();

kId = web3.shh.addPrivateKey(keyPair)
print(web3.shh.hasKeyPair(kId))
print('PubKey: ' + web3.shh.getPublicKey(kId))

#Message sent before filter is assigned
web3.shh.post({
  'pubKey': web3.shh.getPublicKey(kId),
  'topic': '0x07678231',
  'powTarget': 2.01,
  'powTime': 2,
  'ttl': 10,
  'payload': web3.toHex(text="test message :)")

});
time.sleep(1)    

received_messages = [];
myFilter = web3.shh.newMessageFilter({'topic': '0x07678231',
                                      'privateKeyID': kId
                                      });
myFilter.poll_interval = 600;

print(myFilter.filter_id)
received_messages = myFilter.get_all_entries() # Error occurs!!
print(received_messages)

Error I am having:

  File "/home/alper/eBlocBroker/venv/lib/python3.5/site-packages/web3/utils/filters.py", line 191, in get_all_entries
    raise NotImplementedError()
NotImplementedError
alper
  • 8,395
  • 11
  • 63
  • 152

1 Answers1

1

Latest Web3.js does not have web3.shh.getMessages() ; but web3.py has it. So I was able to solve this problem using web3.py.

Shh.getMessages(self, filter_id)

=> Retrieve messages that match the filter criteria and are received between the last time this function was called and now.
=> Returns all new messages since the last invocation

First you have to save keyPair and filter-id:

from web3 import Web3, HTTPProvider
web3 = Web3(HTTPProvider('http://localhost:8545'))
from web3.shh import Shh
Shh.attach(web3, "shh")
import time

keyPair =  web3.shh.newKeyPair();
kId     = web3.shh.addPrivateKey(keyPair)
myFilter = web3.shh.newMessageFilter({'topic': '0x07678231',
                                      'privateKeyID': kId
                                      });
myFilter.poll_interval = 600;
print('keyPair: '  + keyPair)
print('FilterID: ' + myFilter.filter_id)

Output:

keyPair: 0xa82ff6abcf75393084edb06d100e53268c5e118884d59282ef9d3396c8537011
FilterID: 35e2f4c4451fb36da13260c67fe57bdff1f2aeea2bdd346611eef61903c19aaa

Later do following; this will print messages that exists on the memory.:

from web3 import Web3, HTTPProvider
web3 = Web3(HTTPProvider('http://localhost:8545'))
from web3.shh import Shh
Shh.attach(web3, "shh")
import time
print(web3.shh.info)

keyPair =  '0xa82ff6abcf75393084edb06d100e53268c5e118884d59282ef9d3396c8537011'; # obtained from web3.shh.newKeyPair();    
kId = web3.shh.addPrivateKey(keyPair)
print(web3.shh.hasKeyPair(kId))
print('PubKey: ' + web3.shh.getPublicKey(kId))

# Message is sent before the filter is activated
web3.shh.post({
  'pubKey': web3.shh.getPublicKey(kId),
  'topic': '0x07678231',
  'powTarget': 2.01,
  'powTime': 2,
  'ttl': 10,
  'payload': web3.toHex(text="test message :)")

});
time.sleep(1)

filterID = '35e2f4c4451fb36da13260c67fe57bdff1f2aeea2bdd346611eef61903c19aaa' #Filter-id is obtained from previous session.
retreived_messages = web3.shh.getMessages(filterID) 

for i in range(0, len(retreived_messages)):
    print(retreived_messages[i]['payload'].decode("utf-8"))
    print(retreived_messages[i])

Output:

test message :)
{'padding': HexBytes('0xcd6e2ee5fb8d7c4b7e296397c2e9d6d2ba9563c003295004afe488dffc384833b8f5ada10a01dd08f16bf6e72a1f6a6ab45ffa3a1a2e23e2a5853b7978d438e306660e22b3edf93359c290dcf5669072aba1c19ab9012d8904f80f7daa256d0d796f4ae096b7ece2c3a34e2f19f5162916bd4537492a137e84d2b717a3b2a91cf4f6291dd5c26d17cdc7df94ddfdedeaad2b216925e9404c8f645cee9cf338957c374e4d6770f27db8624689b7ba4604b3f05ba7d4fa06560a165860dbd1b54562bde7e96686c96aee7a9ad45e2669a05323eec89f5ea23dcda8308cb494d774a1d28b0f8e4058c410326a8fdf'), 
'topic': HexBytes('0x07678231'), 
'payload': HexBytes('0x74657374206d657373616765203a293e3e'),
'recipientPublicKey': HexBytes('0x04d96278caa49aff29d76399001a43a9a8005da21c6853d20d6af11d61df8a2a25de0fe79a1099bd4b915445d94cbd448b2fe548fb53fed5d887291f35de0f44c9'),
'timestamp': 1530126289, 
'ttl': 10, 
'pow': 8.402051282051282, 
'hash': HexBytes('0x58240972265ea90d3cd8af3d71a8eb4663ae5bfccd00f3e2fb100c5072798017')}
alper
  • 8,395
  • 11
  • 63
  • 152