forked from somatic-labs/meteorite
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathP2P storms (5).eml
325 lines (316 loc) · 15 KB
/
P2P storms (5).eml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
MIME-Version: 1.0
Date: Sun, 24 Sep 2023 08:55:17 +0800
References: <CAK5+0FTnKtgdZdFOND+GYvAcL7wKjxvSn=Z8YRVbXXOhsxKRQA@mail.gmail.com>
<CAGYWHoZPQZnoHg6dg3wJMkD_LLZ0xv9edmEfFz8fcnTvx2_Rdw@mail.gmail.com>
<CAK5+0FTvx_RguvRJQvwo0bOhVcnBz3haz=WTfbyD8qmnOTQt_w@mail.gmail.com>
<CAK5+0FRw+HStw8_b9XfcNLsU8r7ruB6wYyD4Mp6DV7_M7dFhoQ@mail.gmail.com>
<CAGYWHobbbAyYkjCKpUJhVWK1nKTAP3RTf=-8uEWexwNtKEkQ4w@mail.gmail.com>
In-Reply-To: <CAGYWHobbbAyYkjCKpUJhVWK1nKTAP3RTf=-8uEWexwNtKEkQ4w@mail.gmail.com>
Message-ID: <CAK5+0FTWWJ9xGMQenVH1dO4uDdD=GFbRtNmqmduRcZJVLiboGQ@mail.gmail.com>
Subject: Re: P2P storms
From: Jacob Gadikian <[email protected]>
To: Moshe Mizrahi <[email protected]>
Content-Type: multipart/alternative; boundary="0000000000009c9e160606104b98"
--0000000000009c9e160606104b98
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
https://github.com/notional-labs/spammy/invitations
On Sun, Sep 24, 2023, 6:01 AM Moshe Mizrahi <[email protected]> wrote:
> Hey Jacob,
>
> Can you add my handle to the repo? (mizmo18)
>
> Thanks,
> -mo
>
> On Sat, Sep 23, 2023 at 3:38=E2=80=AFAM Jacob Gadikian <[email protected]=
ntures>
> wrote:
>
>> Please also note that there is no field length validation in IBC.
>>
>> If banana king allows for oversized IBC transfer transactions which are
>> valid but still have a cost, the banana client allows for no cost oversi=
zed
>> client updates, which could then be broadcast from several addresses
>> simultaneously and continuously until the p2p Strom is triggered.
>>
>> Banana King is a 100% proven and repeatedly used Way to make transaction=
s
>> that in the wild have been up to 10 megabytes.
>>
>> But IBC client updates on the cosmos hub are free and malformed client
>> updates are still gossiped.
>>
>>
>>
>> On Sat, Sep 23, 2023, 6:31 PM Jacob Gadikian <[email protected]>
>> wrote:
>>
>>> This has been experienced on the following mainnets:
>>>
>>> * stride
>>> * Luna classic
>>> * Sentinel
>>>
>>>
>>> I will need GitHub usernames for team members that you would like on th=
e
>>> repository where I have transactions prepared as JSON.
>>>
>>> That repo, as mentioned in the document is
>>>
>>> GitHub.com/notional-labs/spammy
>>>
>>> On Sat, Sep 23, 2023, 2:44 PM Moshe Mizrahi <[email protected]> wrote:
>>>
>>>> Hey Jacob,
>>>>
>>>> Thanks for taking the time to report this finding. Our team has starte=
d
>>>> the initial triage and efforts to independently reproduce the issue yo=
u=E2=80=99ve
>>>> outlined.
>>>>
>>>> If you have a proof of concept readily available in addition to your
>>>> submission, that would help our team expedite the analysis.
>>>>
>>>> We=E2=80=99ll provide an update once the issue has been triaged, likel=
y
>>>> early next week.
>>>>
>>>> Thanks,
>>>> -mo
>>>>
>>>> On Fri, Sep 22, 2023 at 8:09=E2=80=AFPM Jacob Gadikian <jacob@notional=
.ventures>
>>>> wrote:
>>>>
>>>>> NOTE: THIS DETAILS AN ONGOING MATTER.
>>>>>
>>>>> I'd like to politely urge that Amulet re-join the slack channel that
>>>>> Amulet left so that Amulet can get sufficient context.
>>>>>
>>>>>
>>>>> Greetings,
>>>>>
>>>>> Here is a document on what I am calling "p2p storms". They are a
>>>>> threat to liveness on 100% of cosmos-sdk chains to my knowledge.
>>>>>
>>>>> The document is under heavy revision because I believe that there are
>>>>> at least two ways to cause a p2p storm:
>>>>>
>>>>>
>>>>> - Banana King
>>>>> - Spamming Client Updates
>>>>>
>>>>>
>>>>>
>>>>> https://docs.google.com/document/d/1oCjsVYMaV77etxOEbDxh58vkAQaXf7RAk=
hXvF_8GYis/edit?usp=3Dsharing
>>>>>
>>>>> For a relatively low cost, or in some cases at no cost, it is possibl=
e
>>>>> to DOS entire chains in ways that reduce liveness significantly.
>>>>>
>>>>> Reproducing this issue may not be viable on test networks, as the cor=
e
>>>>> of the issue is the increased p2p traffic that large blocks cause, an=
d a
>>>>> resulting inability of the chain to process its mempool.
>>>>>
>>>>> *P2P storms have been observed in the wild*
>>>>>
>>>>> - On Stride
>>>>> - On Sentinel
>>>>>
>>>>> There's anecdotal evidence of p2p storms on Luna Classic *during* the
>>>>> combined economic and infrastructure attack that took the chain down,=
and
>>>>> you can find my documentation of this here:
>>>>>
>>>>>
>>>>> https://github.com/notional-labs/notional/blob/master/incidents/WTF%2=
0HAPPENED%20TO%20TERRA.pdf
>>>>>
>>>>> Thus far, we've been coordinating in Slack channels, which I'm happy
>>>>> to re-invite you to, since you left because that doesn't fit within y=
our
>>>>> protocol for dealing with security issues.
>>>>>
>>>>> I've been coordinating with Jehan and I'm about to put a Banana King
>>>>> on hub mainnet. A few blocks of banana king is enough to trigger a p=
2p
>>>>> storm, as is a few blocks of client update spam. In fact on further
>>>>> consideration, I think that the LUNC incident is no longer incidental=
.
>>>>>
>>>>> I think that LUNC suffered a p2p storm during the attack, and that th=
e
>>>>> p2p storm was able to enable arbitrage, which is the entire threat mo=
del
>>>>> for this attack.
>>>>>
>>>>> *Recommendations*
>>>>>
>>>>> 1) drop sentry node architecture -> to reduce the number of hops and
>>>>> nodes
>>>>>
>>>>> 2) validators connect to one another over vpn -> same goal as number
>>>>> one
>>>>>
>>>>> 3) reduce maximum block sizes <- I suggest 5mb
>>>>>
>>>>> I need to update the document to add information about banana king.
>>>>> Both client spamming and banana king can trigger p2p storms, and p2p =
storms
>>>>> can be arbed, and have been.
>>>>>
>>>>> --
>>>>> *Jacob Gadikian*
>>>>> CEO
>>>>> *Notional Labs*
>>>>>
>>>>>
>>>>> *Github <https://github.com/faddat> - Twitter
>>>>> <https://twitter.com/gadikian>*
>>>>> *Web: Notional.Ventures <https://notional.ventures/>*
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "security" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, sen=
d
>>>>> an email to [email protected].
>>>>>
>>>>
--0000000000009c9e160606104b98
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"auto"><a href=3D"https://github.com/notional-labs/spammy/invita=
tions">https://github.com/notional-labs/spammy/invitations</a></div><br><di=
v class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Sun, Sep 2=
4, 2023, 6:01 AM Moshe Mizrahi <<a href=3D"mailto:[email protected]">mo@amul=
et.dev</a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"m=
argin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"a=
uto">Hey Jacob,</div><div dir=3D"auto"><br></div><div dir=3D"auto">Can you =
add my handle to the repo? (mizmo18)</div><div dir=3D"auto"><br></div><div =
dir=3D"auto">Thanks,</div><div dir=3D"auto">-mo</div><div><br><div class=3D=
"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Sat, Sep 23, 2023 at=
3:38=E2=80=AFAM Jacob Gadikian <[email protected]> wrote:<br><=
/div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bo=
rder-left-width:1px;border-left-style:solid;padding-left:1ex;border-left-co=
lor:rgb(204,204,204)"><div dir=3D"auto">Please also note that there is no f=
ield length validation in IBC.<div dir=3D"auto"><br></div><div dir=3D"auto"=
>If banana king allows for oversized IBC transfer transactions which are va=
lid but still have a cost, the banana client allows for no cost oversized c=
lient updates, which could then be broadcast from several addresses simulta=
neously and continuously until the p2p Strom is triggered.=C2=A0</div><div =
dir=3D"auto"><br></div><div dir=3D"auto">Banana King is a 100% proven and r=
epeatedly used Way to make transactions that in the wild have been up to 10=
megabytes.</div><div dir=3D"auto"><br></div><div dir=3D"auto">But IBC clie=
nt updates on the cosmos hub are free and malformed client updates are stil=
l gossiped.</div><div dir=3D"auto"><br></div><div dir=3D"auto"><br></div></=
div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On=
Sat, Sep 23, 2023, 6:31 PM Jacob Gadikian <[email protected]> =
wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0=
px 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;bor=
der-left-color:rgb(204,204,204)"><div dir=3D"auto">This has been experience=
d on the following mainnets:<div dir=3D"auto"><br></div><div dir=3D"auto">*=
stride</div><div dir=3D"auto">* Luna classic</div><div dir=3D"auto">* Sent=
inel</div><div dir=3D"auto"><br></div><div dir=3D"auto"><br></div><div dir=
=3D"auto">I will need GitHub usernames for team members that you would like=
on the repository where I have transactions prepared as JSON.</div><div di=
r=3D"auto"><br></div><div dir=3D"auto">That repo, as mentioned in the docum=
ent is</div><div dir=3D"auto"><br></div><div dir=3D"auto">GitHub.com/notion=
al-labs/spammy</div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" c=
lass=3D"gmail_attr">On Sat, Sep 23, 2023, 2:44 PM Moshe Mizrahi <<a href=
=3D"mailto:[email protected]" rel=3D"noreferrer noreferrer noreferrer" target=
=3D"_blank">[email protected]</a>> wrote:<br></div><blockquote class=3D"gmai=
l_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px;border-lef=
t-style:solid;padding-left:1ex;border-left-color:rgb(204,204,204)"><div dir=
=3D"auto">Hey Jacob,</div><div dir=3D"auto"><br></div><div dir=3D"auto">Tha=
nks for taking the time to report this finding. Our team has started the in=
itial triage and efforts to independently reproduce the issue you=E2=80=99v=
e outlined.=C2=A0</div><div dir=3D"auto"><br></div><div dir=3D"auto">If you=
have a proof of concept readily available in addition to your submission, =
that would help our team expedite the analysis.</div><div dir=3D"auto"><br>=
</div><div dir=3D"auto">We=E2=80=99ll provide an update once the issue has =
been triaged, likely early=C2=A0next week.=C2=A0</div><div dir=3D"auto"><br=
></div><div dir=3D"auto">Thanks,</div><div dir=3D"auto">-mo</div><div><br><=
div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Fri, Sep=
22, 2023 at 8:09=E2=80=AFPM Jacob Gadikian <[email protected]>=
wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px =
0px 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;bo=
rder-left-color:rgb(204,204,204)"><div dir=3D"ltr"><div>NOTE: THIS DETAILS =
AN ONGOING MATTER.</div><div><br></div><div>I'd like to politely urge t=
hat Amulet re-join the slack=C2=A0channel that Amulet left so that Amulet c=
an get sufficient=C2=A0context.=C2=A0</div><div><br></div><div><br></div>Gr=
eetings,<div><br></div><div>Here is a document on what I am calling "p=
2p storms".=C2=A0 They are a threat to liveness on 100% of cosmos-sdk =
chains to my knowledge.=C2=A0=C2=A0</div><div><br></div><div>The document i=
s under heavy revision because I believe that there are at least two ways t=
o cause a p2p storm:</div><div><br></div><div><ul><li>Banana King</li><li>S=
pamming Client Updates</li></ul></div><div><br></div><div><a href=3D"https:=
//docs.google.com/document/d/1oCjsVYMaV77etxOEbDxh58vkAQaXf7RAkhXvF_8GYis/e=
dit?usp=3Dsharing" rel=3D"noreferrer noreferrer noreferrer noreferrer" targ=
et=3D"_blank">https://docs.google.com/document/d/1oCjsVYMaV77etxOEbDxh58vkA=
QaXf7RAkhXvF_8GYis/edit?usp=3Dsharing</a></div><div><br></div><div>For a re=
latively low cost, or in some cases at no cost, it is possible to DOS entir=
e chains in ways that reduce liveness significantly.=C2=A0</div><div><br></=
div><div>Reproducing this issue may not be viable on test networks, as the =
core of the issue is the increased p2p traffic that large blocks cause, and=
a resulting inability of the chain to process its mempool.=C2=A0=C2=A0</di=
v><div><br></div><div><b>P2P storms have been observed in the wild</b></div=
><div><ul><li>On Stride</li><li>On Sentinel</li></ul><div>There's anecd=
otal evidence of p2p storms on Luna Classic <b>during</b>=C2=A0the combined=
economic=C2=A0and infrastructure=C2=A0attack that took the chain down, and=
you can find my documentation of this here:</div></div><div><br></div><div=
><a href=3D"https://github.com/notional-labs/notional/blob/master/incidents=
/WTF%20HAPPENED%20TO%20TERRA.pdf" rel=3D"noreferrer noreferrer noreferrer n=
oreferrer" target=3D"_blank">https://github.com/notional-labs/notional/blob=
/master/incidents/WTF%20HAPPENED%20TO%20TERRA.pdf</a><br></div><div><br></d=
iv><div>Thus far, we've been coordinating in Slack channels, which I=
9;m happy to re-invite you to, since you left because that doesn't fit =
within your protocol for dealing with security issues.=C2=A0=C2=A0</div><di=
v><br></div><div>I've been coordinating with Jehan and I'm about to=
put a Banana King on hub mainnet.=C2=A0 A few blocks of banana king is eno=
ugh to trigger a p2p storm, as is a few blocks of client update spam.=C2=A0=
In fact on further consideration, I think that the LUNC=C2=A0incident is n=
o longer incidental.=C2=A0=C2=A0</div><div><br></div><div>I think that LUNC=
=C2=A0suffered a p2p storm during the attack, and that the p2p storm was ab=
le to enable arbitrage, which is the entire threat model for this attack.=
=C2=A0</div><div><br></div><div><b>Recommendations</b></div><div><br></div>=
<div>1) drop sentry node architecture=C2=A0 -> to reduce the number of h=
ops and nodes <br><br>2) validators connect to one another over vpn -> s=
ame goal as number one<br><br>3) reduce maximum block sizes <- I suggest=
5mb=C2=A0<br></div><div><br></div><div>I need to update the document to ad=
d information about banana king.=C2=A0 Both client spamming and banana king=
can trigger p2p storms, and p2p storms can be arbed, and have been.=C2=A0<=
/div></div><div dir=3D"ltr"><div><div><br></div><span class=3D"gmail_signat=
ure_prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature" data-s=
martmail=3D"gmail_signature"><div dir=3D"ltr"><b>Jacob Gadikian</b><div>CEO=
</div><div><b>Notional Labs</b></div><div><b><br></b></div><div><b><img wid=
th=3D"200" height=3D"45" src=3D"https://ci3.googleusercontent.com/mail-sig/=
AIorK4wD9P5t4zjYRMBeM-6EevwBrTlygoGMP7t4XFQdD-pdpfcJHseOCfENf_YepQEMpsyIXNE=
SHHs"><br></b></div><div><b><a href=3D"https://github.com/faddat" rel=3D"no=
referrer noreferrer noreferrer noreferrer" target=3D"_blank">Github</a> - <=
a href=3D"https://twitter.com/gadikian" rel=3D"noreferrer noreferrer norefe=
rrer noreferrer" target=3D"_blank">Twitter</a></b></div><div><b>Web: <a hre=
f=3D"https://notional.ventures/" rel=3D"noreferrer noreferrer noreferrer no=
referrer" target=3D"_blank">Notional.Ventures</a></b></div></div></div></di=
v></div>
<p></p>
-- <br>
You received this message because you are subscribed to the Google Groups &=
quot;security" group.<br>
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:[email protected]" rel=3D"norefe=
rrer noreferrer noreferrer noreferrer" target=3D"_blank">security+unsubscri=
[email protected]</a>.<br>
</blockquote></div></div>
</blockquote></div>
</blockquote></div>
</blockquote></div></div>
</blockquote></div>
--0000000000009c9e160606104b98--