Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

all etherscan #23

Closed
wants to merge 27 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
5be6805
puppeteer scrape address page properly
dawsbot Feb 5, 2024
514baa8
scrape account pages effectively
dawsbot Feb 5, 2024
429c95a
Add todo
dawsbot Mar 6, 2024
de8526c
Copy readme over from main
dawsbot Mar 6, 2024
7261b25
Update contributing readme
dawsbot Mar 6, 2024
135ae11
fix error parsing and add dynamic hostname generation (#18)
kylewandishin Mar 9, 2024
3715502
working log in info from dotenv. scraping pages of 25 frmo each label
kylewandishin Mar 12, 2024
75588f1
added automatic sign in and scraping capabilities for signed in user\…
kylewandishin Mar 12, 2024
b9780a2
scrape every account 'main' labels into json files
kylewandishin Mar 12, 2024
e94f3b7
pull all main accounts from etherscan. parse as json in /data\n\nclos…
kylewandishin Mar 12, 2024
3638263
add runtime automated testing for scraping and speed up pulling proce…
kylewandishin Mar 19, 2024
d008efb
fix timeout
kylewandishin Mar 19, 2024
47ebe10
data checkpoint
kylewandishin Mar 19, 2024
4618c6a
sorting added
kylewandishin Mar 19, 2024
acbebef
sorted data
kylewandishin Mar 19, 2024
ac57f73
rename data folders
kylewandishin Mar 19, 2024
0466900
pull data from all labels with all subcatagories. depreciates length …
kylewandishin Mar 19, 2024
f628968
format all code files
kylewandishin Mar 19, 2024
235e770
clean up workspace
kylewandishin Mar 19, 2024
c0f9dd6
turned etherscan pulling into a component to begin refactoring
kylewandishin Mar 19, 2024
1832675
turned etherscan-pulling into a component to begin refactoring
kylewandishin Mar 19, 2024
2d93cfc
restyle code with prettier
kylewandishin Mar 19, 2024
4961767
Move etherscan to class and add eslint
dawsbot Mar 20, 2024
e4b0073
Fresh pull without lowercase re-ordering
dawsbot Mar 20, 2024
450d9ca
improve function names
dawsbot Mar 20, 2024
2c5186f
Pull accounts
dawsbot Mar 24, 2024
cd1e4b3
Sort all tokens, improve logging
dawsbot Mar 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
18 changes: 18 additions & 0 deletions .eslintrc.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
module.exports = {
ignorePatterns: ["lib/**", "**/*.js"],
parser: "@typescript-eslint/parser",
plugins: ["@typescript-eslint"],
extends: [
"eslint:recommended",
"plugin:@typescript-eslint/recommended-type-checked",
"prettier",
],
parserOptions: {
project: ["./tsconfig.json"],
// tsconfigRootDir: __dirname,
// project: ['./tsconfig.base.json', './packages/*/tsconfig.json'], // Specify it only for TypeScript files
},
env: {
node: true,
},
};
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,5 @@ node_modules
dist
lib
types
.env
.eslintcache
50 changes: 50 additions & 0 deletions data/etherscan/0x-protocol-ecosystem/accounts.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
[
{
"address": "0x0e8ba001a821f3ce0734763d008c9d7c957f5852",
"nameTag": "AmadeusRelay"
},
{
"address": "0xc898fbee1cc94c0ff077faa5449915a506eff384",
"nameTag": "Bamboo Relay"
},
{
"address": "0x58a5959a6c528c5d5e03f7b9e5102350e24005f1",
"nameTag": "ERC dEX"
},
{
"address": "0x2cc42d1cd65af27cc999e41ef93d1a763dc821f8",
"nameTag": "IDT Exchange"
},
{
"address": "0x4524baa98f9a3b9dec57caae7633936ef96bd708",
"nameTag": "LedgerDex"
},
{
"address": "0xc22d5b2951db72b44cfb8089bb8cd374a3c354ea",
"nameTag": "OpenRelay"
},
{
"address": "0xd2045edc40199019e221d71c0913343f7908d0d5",
"nameTag": "Paradex"
},
{
"address": "0xa258b39954cef5cb142fd567a46cddb31a670124",
"nameTag": "Radar Relay"
},
{
"address": "0x55890b06f0877a01bb5349d93b202961f8e27a9b",
"nameTag": "Shark Relay"
},
{
"address": "0x0681e844593a051e2882ec897ecd5444efe19ff2",
"nameTag": "Star Bit Ex"
},
{
"address": "0x7219612be7036d1bfa933e16ca1246008f38c5fe",
"nameTag": "The Ocean"
},
{
"address": "0x5e150a33ffa97a8d22f59c77ae5487b089ef62e9",
"nameTag": "TokenJar"
}
]
68 changes: 68 additions & 0 deletions data/etherscan/0x-protocol-ecosystem/tokens.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
[
{
"address": "0x1c74cff0376fb4031cd7492cd6db2d66c3f2c6b9",
"tokenName": "bZx Protocol...",
"tokenSymbol": "BZRX",
"website": "https://bzx.network/"
},
{
"address": "0x221657776846890989a759ba2973e427dff5c9bb",
"tokenName": "Reputation",
"tokenSymbol": "REPv2",
"website": "https://www.augur.net/"
},
{
"address": "0x4de2573e27E648607B50e1Cfff921A33E4A34405",
"tokenName": "Lendroid Sup...",
"tokenSymbol": "LST",
"website": "https://lendroid.com/"
},
{
"address": "0x6531f133e6deebe7f2dce5a0441aa7ef330b4e53",
"tokenName": "TIME",
"tokenSymbol": "TIME",
"website": "https://chrono.tech/"
},
{
"address": "0x701c244b988a513c945973defa05de933b23fe1d",
"tokenName": "OpenANX",
"tokenSymbol": "OAX",
"website": "https://www.openanx.org"
},
{
"address": "0x8f8221afbb33998d8584a2b05749ba73c37a938a",
"tokenName": "Request",
"tokenSymbol": "REQ",
"website": "https://request.network/"
},
{
"address": "0x9f8f72aa9304c8b593d555f12ef6589cc3a579a2",
"tokenName": "Maker",
"tokenSymbol": "MKR",
"website": "https://makerdao.com/"
},
{
"address": "0xb3319f5d18bc0d84dd1b4825dcde5d5f7266d407",
"tokenName": "Compound 0x",
"tokenSymbol": "cZRX",
"website": "https://compound.finance/"
},
{
"address": "0xe41d2489571d322189246dafa5ebde1f4699f498",
"tokenName": "ZRX",
"tokenSymbol": "ZRX",
"website": "https://0x.org/"
},
{
"address": "0xec67005c4e498ec7f55e092bd1d35cbc47c91892",
"tokenName": "Melon Token",
"tokenSymbol": "MLN",
"website": "https://enzyme.finance/"
},
{
"address": "0xf5ed2dc77f0d1ea7f106ecbd1850e406adc41b51",
"tokenName": "The Ocean To...",
"tokenSymbol": "OCEAN",
"website": "https://theocean.trade/"
}
]
234 changes: 234 additions & 0 deletions data/etherscan/0x-protocol/accounts.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,234 @@
[
{
"address": "0xda6d9fc5998f550a094585cf9171f0e8ee3ac59f",
"nameTag": "0x: Affiliate Fee Transformer"
},
{
"address": "0xdffe798c7172dd6deb32baee68af322e8f495ce0",
"nameTag": "0x: Asset Proxy Owner"
},
{
"address": "0xd4690a51044db77d91d7aa8f7a3a5ad5da331af0",
"nameTag": "0x: Broker V3"
},
{
"address": "0xe66b31678d6c16e9ebf358268a790b763c133750",
"nameTag": "0x: Coinbase Wallet Proxy"
},
{
"address": "0x45797531b873fd5e519477a070a955764c1a5b07",
"nameTag": "0x: Coordinator Registry V3"
},
{
"address": "0x38a795580d0f687e399913a00ddef6a17612c722",
"nameTag": "0x: Coordinator V3"
},
{
"address": "0xa3b2d1087bcebe59d188a23f75620612d967df72",
"nameTag": "0x: Deployer"
},
{
"address": "0x2d7dc2ef7c6f6a2cbc3dba4db97b2ddb40e20713",
"nameTag": "0x: Deployer 2"
},
{
"address": "0x3b39078f2a3e1512eecc8d6792fdc7f33e1cd2cf",
"nameTag": "0x: Deployer 3"
},
{
"address": "0xccc2431a7335f21d9268ba62f0b32b0f2efc463f",
"nameTag": "0x: Dev Utils"
},
{
"address": "0x74134cf88b21383713e096a5ecf59e297dc7f547",
"nameTag": "0x: Dev Utils 2"
},
{
"address": "0x161793cdca4ff9e766a706c2c49c36ac1340bbcd",
"nameTag": "0x: Dev Utils V3"
},
{
"address": "0x7eefbd48fd63d441ec7435d024ec7c5131019add",
"nameTag": "0x: ERC1155 Proxy V3"
},
{
"address": "0x8ed95d1746bf1e4dab58d8ed4724f1ef95b20db0",
"nameTag": "0x: Erc20 Bridge Proxy"
},
{
"address": "0xd8c38704c9937ea3312de29f824b4ad3450a5e61",
"nameTag": "0x: ERC20 Bridge Sampler"
},
{
"address": "0x95e6f48254609a6ee006f7d493c8e5fb97094cef",
"nameTag": "0x: ERC20 Proxy V3"
},
{
"address": "0xefc70a1b18c432bdc64b596838b4d138f6bc6cad",
"nameTag": "0x: ERC721 Proxy V3"
},
{
"address": "0x0ac2d6f5f5afc669d3ca38f830dad2b4f238ad3f",
"nameTag": "0x: Eth2Dai Bridge"
},
{
"address": "0xdef1c0ded9bec7f1a1670819833240f027b25eff",
"nameTag": "0x: Exchange Proxy"
},
{
"address": "0xf740b67da229f2f10bcbd38a7979992fcc71b8eb",
"nameTag": "0x: Exchange Proxy Allowance Target"
},
{
"address": "0x22f9dcf4647084d6c31b2765f6910cd85c178c18",
"nameTag": "0x: Exchange Proxy Flash Wallet"
},
{
"address": "0x618f9c67ce7bf1a50afa1e7e0238422601b0ff6e",
"nameTag": "0x: Exchange Proxy Governor"
},
{
"address": "0x407b4128e9ecad8769b2332312a9f655cb9f5f3a",
"nameTag": "0x: Exchange Proxy Liquidity Provider Sandbox"
},
{
"address": "0x39dce47a67ad34344eab877eae3ef1fa2a1d50bb",
"nameTag": "0x: Exchange Proxy Transformer Deployer"
},
{
"address": "0x12459c951127e0c374ff9105dda097662a027093",
"nameTag": "0x: Exchange v1"
},
{
"address": "0x080bf510fcbf18b91105470639e9561022937712",
"nameTag": "0x: Exchange v2.1"
},
{
"address": "0x61935cbdd02287b511119ddb11aeb42f1593b7ef",
"nameTag": "0x: Exchange v3"
},
{
"address": "0x206376e8940e42538781cd94ef024df3c1e0fd43",
"nameTag": "0x: Ext Dev Pool"
},
{
"address": "0xb4fa284689c9784a60d840eb136bb16c5246191f",
"nameTag": "0x: Fill Quote Transformer"
},
{
"address": "0xa3ac9844514b96bb502627ca9dceb57c4be289e3",
"nameTag": "0x: Forwarder 2"
},
{
"address": "0x6958f5e95332d93d21af0d7b9ca85b8212fee0a5",
"nameTag": "0x: Forwarder V3"
},
{
"address": "0x7d3455421bbc5ed534a83c88fd80387dc8271392",
"nameTag": "0x: Governor V3"
},
{
"address": "0xef701d5389ae74503d633396c4d654eabedc9d78",
"nameTag": "0x: Multi Asset Proxy V3"
},
{
"address": "0x606af0bd4501855914b50e2672c5926b896737ef",
"nameTag": "0x: MultiSig"
},
{
"address": "0x01d9f4d104668cdc0b6d13c45dff5e15d58d8f28",
"nameTag": "0x: MultiSig 2"
},
{
"address": "0x17992e4ffb22730138e4b62aaa6367fa9d3699a6",
"nameTag": "0x: Old Asset Proxy Owner"
},
{
"address": "0x2240dab907db71e64d3e0dba4800c83b5c502d4e",
"nameTag": "0x: Old ERC20 Proxy"
},
{
"address": "0x208e41fb445f1bb1b6780d58356e81405f3e6127",
"nameTag": "0x: Old ERC721 Proxy"
},
{
"address": "0x4f833a24e1f95d70f028921e27040ca56e09ab0b",
"nameTag": "0x: Old Exchange v2.0"
},
{
"address": "0x7afc2d5107af94c462a194d2c21b5bdd238709d6",
"nameTag": "0x: Old Forwarder"
},
{
"address": "0x5468a1dc173652ee28d249c271fa9933144746b1",
"nameTag": "0x: Old Forwarder 2"
},
{
"address": "0xdc4587cb17d2a1829512e2cfec621f8066290e6a",
"nameTag": "0x: Old Forwarder 3"
},
{
"address": "0x9463e518dea6810309563c81d5266c1b1d149138",
"nameTag": "0x: Old Order Validator"
},
{
"address": "0xa09329c6003c9a5402102e226417738ee22cf1f2",
"nameTag": "0x: Order Validator"
},
{
"address": "0x4638a7ebe75b911b995d0ec73a81e4f85f41f24e",
"nameTag": "0x: Pay Taker Transformer"
},
{
"address": "0xa9416ce1dbde8d331210c07b5c253d94ee4cc3fd",
"nameTag": "0x: Positive Slippage Fee Transformer"
},
{
"address": "0x2a17c35ff147b32f13f19f2e311446eeb02503f3",
"nameTag": "0x: Staking"
},
{
"address": "0xa26e80e7dea86279c6d778d702cc413e6cffa777",
"nameTag": "0x: Staking Proxy V3"
},
{
"address": "0x3517b88c19508c08650616019062b898ab65ed29",
"nameTag": "0x: Static Call Proxy V3"
},
{
"address": "0xdb63d40c033d35e79cdbb21430f0fe10e9d97303",
"nameTag": "0x: Team Vesting"
},
{
"address": "0x926a74c5c36adf004c87399e65f75628b0f98d2c",
"nameTag": "0x: Token Registry"
},
{
"address": "0xd4fd252d7d2c9479a8d616f510eac6243b5dddf9",
"nameTag": "0x: Token Sale"
},
{
"address": "0x8da0d80f5007ef1e431dd2127178d224e32c2ef4",
"nameTag": "0x: Token Transfer Proxy"
},
{
"address": "0xa6baaed2053058a3c8f11e0c7a9716304454b09e",
"nameTag": "0x: Uniswap Bridge"
},
{
"address": "0x06adaff9c864eb59ade310bc21150a8f4d5d4ab4",
"nameTag": "0x: WBTC Merchant Deposit Address"
},
{
"address": "0xb2bc06a4efb20fc6553a69dbfa49b7be938034a7",
"nameTag": "0x: WETH Transformer"
},
{
"address": "0xe41d2489571d322189246dafa5ebde1f4699f498",
"nameTag": "0x: ZRX Token"
},
{
"address": "0xba7f8b5fb1b19c1211c5d49550fcd149177a5eaf",
"nameTag": "0x: ZRX Vault V3"
}
]
8 changes: 8 additions & 0 deletions data/etherscan/0x-protocol/tokens.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[
{
"address": "0xe41d2489571d322189246dafa5ebde1f4699f498",
"tokenName": "ZRX",
"tokenSymbol": "ZRX",
"website": "https://0x.org/"
}
]
Loading