Releases: hashgraph/hedera-mirror-node
v0.24.0-rc1
Bump versions for v0.23.0-rc1 (#1318) Signed-off-by: Steven Sheehy <[email protected]>
v0.23.1
v0.23.0
This release focuses on finalizing our support for the new Hedera Token Service (HTS) provided by the Hedera API v0.9.0. There are no new HTS features, just various fixes to make it compatible with the latest protobuf. HTS is currently enabled in previewnet and should be enabled in testnet very soon, so please try it out and let us know if you have any feedback.
A new Helm chart was added to run the monitor application. The monitor is still under heavy development so stay tuned.
Most of the other changes were bug fixes. We now use SonarCloud to scan for vulnerabilities and bugs and have addressed all the major items. You can view our SonarCloud dashboard to track our progress. Entities are now only inserted for successful transactions and we fixed the wrong address book being updated. There were multiple issues with the state proof alpha API that were resolved. For the gRPC API, we improved its performance and lowered its CPU usage. Also related to gRPC, we now enable server sent keep alive messages and permit a lower client sent keep alive messages of 90 seconds, which should hopefully address timeout issues that some users have reported.
Enhancements
- Bump version for 0.23.0 #1280
- Bump versions for 0.23.0-rc1 #1275
- Bump sdk and update token and crypto transfer transactions #1260
- Change token symbol size to 100 #1258
- Token HAPI changes #1256
- Change token symbol size to 100 #1255
- Refactor importer to handle token transfers in crypto transfer #1254
- Add monitor charts #1251
- Enable SonarCloud code scanning #1230
- Cache current address book #1226
- Increase gRPC server keepalive #1223
- Add new monitor to Kubernetes #1216
- HTS: Support Hbar transfers during token transfer transaction #1176
- Enable SonarCloud #1175
- Cache the current address book #996
- Consider increasing frequency of GRPC server keepalive #802
Bug Fixes
- Bump monitor dashboard dependencies #1276
- Fix incorrect base64 encoding in stateproof #1263
- Fix SonarCloud security hotspots #1262
- Rest State Proof retrieves different record files #1261
- Fix state proof check failure on failed transaction check #1259
- Monitor state proof check failure #1257
- Fix SonarCloud bugs #1253
- Fix NoSuchMethodError during hcs-perf-subscribe-test #1252
- Fix address book update always updating 102 file #1236
- Add version migrations to fix entity type mismatches in db #1235
- Upgrade Spring Boot to 2.3.5 to fix pgpool connection issue #1234
- Fix gRPC performance bottleneck #1232
- Performance bottleneck most flux operations in SharedTopicListener are run in the same thread #1231
- AddressBookService updatePreviousAddressBook only updates 102 file #1229
- Ensure entities are inserted only on create #1228
- Importer incorrectly persists non existent entity on ConsensusSubmitMessage #1201
Dependency Upgrades
- Bump software.amazon.awssdk:bom from 2.15.9 to 2.15.23 #1248
- Bump testcontainers-java from 1.14.3 to 1.15.0 #1247
- Bump jest from 26.6.2 to 26.6.3 in /hedera-mirror-rest #1245
- Bump sinon from 9.2.0 to 9.2.1 in /hedera-mirror-rest #1244
- Bump swagger-stats from 0.95.17 to 0.95.18 in /hedera-mirror-rest #1243
- Bump supertest from 5.0.0 to 6.0.1 in /hedera-mirror-rest #1242
- Bump mathjs from 7.6.0 to 8.0.0 in /hedera-mirror-rest/monitoring/monitor_apis #1241
- Bump eslint-config-airbnb-base from 14.2.0 to 14.2.1 in /hedera-mirror-rest #1240
- Bump jest from 26.6.1 to 26.6.3 in /hedera-mirror-rest/monitoring/monitor_apis #1239
- Bump supertest from 6.0.0 to 6.0.1 in /hedera-mirror-rest/monitoring/monitor_apis #1238
Contributors
We'd like to thank all the contributors who worked on this release!
v0.23.0-rc1
Bump versions for 0.23.0-rc1 (#1275) Signed-off-by: Steven Sheehy <[email protected]>
v0.22.0
This release continues our improvements to our Kubernetes support as well as monitoring and performance improvements across the modules.
We improved our custom PrometheusRule alerts for the Importer, GRPC and REST API modules, as well as added dashboards for our gRPC and REST API modules. Additionally, we increased our pod resources limits to optimize Importer ingestion and gRPC streaming performance in a Kubernetes cluster. Our existing js based monitor and REST performance tests were both updated to include HTS support.
We also improved our data generator module with support for the majority of HAPI transactions the mirror node importer ingests. Additionally, we also added a java based monitor module that supports generation and publishing of transactions.
This release also includes an improvements to avoid the stale account info bug that stems from balance stream files being received at a slower frequency than record stream files. Now account creations and account info changes will be reflected in REST API call even though the updated balance may not have been received.
We also extended our REST API support to include case insensitive support query parameters. /api/v1/transactions?transactionType=tokentransfers and /api/v1/transactions?transactiontype=tokentransfers are now both acceptable.
Enhancements
- Bump versions for v0.22.0-rc1 #1225
- Add a Java based monitor #1203
- Increase importer and rest container resource limit #1202
- Add Transaction Suppliers to generate HAPI transactions #1199
- HTS: Add HTS REST API tests to monitor #1197
- Add custom alerts and dashboards #1170
- Add support for case insensitive query params to REST API #1168
- Update acceptance tests with v0.21.0 token rest checks #1167
- Optimize AccountBalanceRepository queries #1163
- AccountBalanceRepository find* methods are extremely slow #1162
- REST API query params should be case insensitive #1159
- Acceptance Tests should support Token REST APIs added in v0.21.0 #1152
- HTS: Add to monitor #1150
- HTS REST Performance Tests #1135
- Kubernetes: Application Dashboards #660
Bug Fixes
- Fix mainnet bucket name #1222
- Add redis sentinel configuration for importer #1204
- Fix ImporterParseLatency alerts triggering in dev repeatedly #1198
- Fix test case fragmentedMessagesGroupAcrossHistoricAndIncoming #1196
- Fix bad PrometheusRule expression #1182
- Fix stale accounts REST API #1181
- Disable failing ConsensusControllerTest.fragmentedMessagesGroupAcross… #1180
- Failing ConsensusControllerTest in GitHub Action #1178
- Accounts Info REST API responses are often stale #1177
- Add missing columns to token info REST API #1171
- Support disabling pgpool load balancing in importer #1166
- Fix postgresql deadlock in integration tests #1165
- Postgresql deadlock in importer integration test #1164
- Kubernetes: postgresql-ha replication delay causes importer hash mismatch #1154
Dependency Upgrades
- Bump jest from 26.5.3 to 26.6.2 in /hedera-mirror-rest #1215
- Bump grpc.version from 1.33.0 to 1.33.1 #1214
- Bump cucumber.version from 6.8.1 to 6.8.2 #1213
- Bump jmeter-maven-plugin from 3.1.1 to 3.1.2 #1212
- Bump embedded.testcontainers.version from 1.84 to 1.85 #1211
- Bump eslint-config-prettier from 6.14.0 to 6.15.0 in /hedera-mirror-rest #1210
- Bump testcontainers from 4.3.0 to 5.0.0 in /hedera-mirror-rest #1209
- Bump jest from 26.5.3 to 26.6.1 in /hedera-mirror-rest #1208
- Bump pg from 8.4.1 to 8.4.2 in /hedera-mirror-rest #1207
- Bump supertest from 5.0.0 to 6.0.0 in /hedera-mirror-rest/monitoring/monitor_apis #1206
- Bump
@awaitjs/express from 0.6.1 to 0.6.3 in /hedera-mirror-rest #1205 - Bump grpc-netty-shaded from 1.30.2 to 1.33.0 #1194
- Bump embedded.testcontainers.version from 1.83 to 1.84 #1193
- Bump spring-boot-starter-parent from 2.3.3.RELEASE to 2.3.4.RELEASE #1192
- Bump grpc.version from 1.31.1 to 1.33.0 #1191
- Bump guava from 29.0-jre to 30.0-jre #1190
- Bump mathjs from 7.3.0 to 7.5.1 in /hedera-mirror-rest #1189
- Bump jest from 26.5.3 to 26.6.1 in /hedera-mirror-rest/monitoring/monitor_apis #1188
- Bump nodemon from 2.0.5 to 2.0.6 in /hedera-mirror-rest #1187
- Bump eslint-config-prettier from 6.12.0 to 6.14.0 in /hedera-mirror-rest #1186
- Bump nodemon from 2.0.5 to 2.0.6 in /hedera-mirror-rest/monitoring/monitor_apis #1185
- Bump aws-sdk from 2.773.0 to 2.778.0 in /hedera-mirror-rest #1184
- Bump axios from 0.20.0 to 0.21.0 in /hedera-mirror-rest #1183
Contributors
We'd like to thank all the contributors who worked on this release!
v0.22.0-rc1
Bump versions for v0.22.0-rc1 (#1225) * Bump v0.22.0-rc1 Bump versions for v0.22.0-rc1 and release char version to v0.9.0-rc1 Signed-off-by: Nana-EC <[email protected]>
v0.21.0
This release continues our focus on the Hedera Token Service (HTS) by adding three new token REST APIs. A token discovery REST API /api/v1/tokens shows available tokens on the network. A token REST API /api/v1/tokens/${tokenId} shows details for a particular token on the network. A token supply distribution REST API /api/v1/tokens/${tokenId}/balances shows token distribution across accounts. These APIs have already made their way to previewnet so check them out!
Continuing our HTS theme, we enhanced our testing frameworks with token support. Our acceptance tests can send HTS transactions to HAPI and wait for those transactions to show up via the mirror node REST API. Additionally, our performance tests can simulate a HTS load to test how the system responds to HTS transactions.
We improved our existing REST APIs by adding a way to filter by transaction type. When searching for transactions or showing the transactions for a particular account you can now filter via an optional transactionType query parameter. This feature can be used with the transactions API in the format /api/v1/transactions?transactionType=tokentransfers while the format for the accounts API is /api/v1/accounts/0.0.8?transactionType=TOKENTRANSFERS.
We improved our Kubernetes support with AlertManager integration. There are now custom PrometheusRule alerts for each component that trigger notifications based upon Prometheus metrics. A custom Grafana dashboard was created that shows currently firing alerts.
Enhancements
- Bump versions for 0.21.0 #1224
- Bump version for v0.21.0-rc1 #1161
- Add transaction type filter to REST API #1139
- HTS e2e Performance Tests #1134
- Acceptance Tests should support initial Token Scenarios added in v0.20.0 #1132
- Updated Acceptance test to support base HTS scenarios #1131
- Add support for alerts in Kubernetes #1130
- HTS: Token REST API #1125
- HTS: Token supply distribution REST API #1113
- HTS: REST Token Discovery API #1110
- HTS: Stability and performance testing #1108
- HTS: Token REST API #1106
- HTS: Token supply distribution REST API #1104
- HTS: Token discovery REST API #1103
- Kubernetes: Alerts #658
Bug Fixes
- Bump v0.21.0-rc2 with missing columns to token info REST API #1173
- modified_timestamp and name are missing from token info REST API response #1172
- Port Fix NPE parsing token transactions to master #1158
- Address code scanning results #1111
Documentation
- Minor updates to HTS doc post REST implementation #1138
Dependency Upgrades
- Bump software.amazon.awssdk:bom from 2.15.7 to 2.15.9 #1148
- Bump commons-io from 2.7 to 2.8.0 #1147
- Bump jackson-dataformat-msgpack from 0.8.20 to 0.8.21 #1146
- Bump pretty-quick from 3.0.2 to 3.1.0 in /hedera-mirror-rest #1145
- Bump aws-sdk from 2.761.0 to 2.773.0 in /hedera-mirror-rest #1144
- Bump jest-junit from 11.1.0 to 12.0.0 in /hedera-mirror-rest #1143
- Bump nodemon from 2.0.4 to 2.0.5 in /hedera-mirror-rest #1142
- Bump codecov from 3.7.2 to 3.8.0 in /hedera-mirror-rest #1141
- Bump nodemon from 2.0.4 to 2.0.5 in /hedera-mirror-rest/monitoring/monitor_apis #1140
- Bump jib-maven-plugin from 2.5.2 to 2.6.0 #1124
- Bump software.amazon.awssdk:bom from 2.13.11 to 2.15.7 #1123
- Bump embedded.testcontainers.version from 1.82 to 1.83 #1121
- Bump mathjs from 7.3.0 to 7.5.1 in /hedera-mirror-rest/monitoring/monitor_apis #1120
- Bump pg from 8.3.3 to 8.4.1 in /hedera-mirror-rest #1119
- Bump jest from 26.4.2 to 26.5.3 in /hedera-mirror-rest/monitoring/monitor_apis #1118
- Bump sinon from 9.1.0 to 9.2.0 in /hedera-mirror-rest #1116
- Bump jest from 26.4.2 to 26.5.3 in /hedera-mirror-rest #1115
- Bump eslint-plugin-jest from 24.0.2 to 24.1.0 in /hedera-mirror-rest #1114
Contributors
We'd like to thank all the contributors who worked on this release!
v0.20.0
This is a big release that contains support for a new HAPI service and whole new runtime component to dramatically improve performance. Due to the magnitude of the changes, it did take us a little longer to mark it as generally available as we wanted to ensure it was tested as much as possible beforehand.
First up is support for the Hedera Token Service (HTS) that was recently announced and rolled out to previewnet. A lot of work was put into supporting the new transaction types on the parser side including enhancing the schema with new tables and ingesting them via the record stream. HTS also required a new balance file version that adds token information to the CSV. Token information is now returned for our existing REST APIs while the next release will contain token specific REST APIs for further granularity. Check it out in previewnet and let us know if you have any feedback!
We made a lot of strides in improving the ingestion performance in previous releases, but since we also wanted to ensure low end to end HCS latency via our gRPC API we had to sacrifice some of that speed. As a result, we could only ingest at about 3,000 transactions per second (TPS) before latency spiked above 10 seconds. This was entirely due to our use of PostgreSQL notify/listen to notify the gRPC API of new data.
In this release, we add a new notification mechanism without sacrificing speed or latency with our support for Redis pub/sub. With Redis, the mirror node can now ingest at least 10,000 TPS while still remaining under 10 seconds from submitting the topic message and receiving it back via the mirror node's streaming API. Redis is enabled by default, but it can be turned off if HCS latency is not a concern and you want to avoid another runtime dependency.
We also added support for the HAPI protobuf changes that are coming in v0.9.0. The protobuf is removing two deprecated fields while adding a new signedTransactionBytes field. Since the mirror node still needs to support historical transactions we retain support for parsing transactions that contain the old payload format.
Enhancements
- Bump versions for 0.20.0 #1200
- Bump version to rc1 from alpha1 w HTS Acceptance Tests #1137
- Bump versions for v0.20.0-alpha1 #1109
- Enable code scanning for vulnerabilities #1092
- Add Redis support to Helm #1091
- HTS: Support new balance file version #1090
- HTS: Add support for token transactions to Importer #1089
- HTS: Add token balances to accounts REST API #1088
- HTS: Add new fields to accounts, balances and transactions REST API #1087
- HTS: Minor updates to design and schema #1086
- Add topic messages and stateproof to monitor #1076
- Add a Redis HCS topic listener #1075
- stateproof: download record file from stored node #1074
- HTS: DB schema changes #1073
- Replace account_realm_num and account_num with account_id in account_balance table #1071
- Migrate account_balance table to use entity_id for account id #1070
- State proof alpha: download record file from the node used to verify and download the file in importer #1069
- HTS: Design document #1061
- allow individual maxLimit config in rest service monitor #1060
- Bump hedera-protobuf to 0.9.0-alpha2 and support new fields #1056
- HTS: Import token transactions #1051
- HTS: Support new balance file version #1049
- HTS: Database Schema #1048
- HTS: Add new fields to balances and transactions REST API #1047
- HTS: Design #1043
- Allow for configuration of specific checks on the monitor #1033
- Protobuf changes for 0.9.0 #1027
- Pre-OA Mirror Node #913
- Reduce subscribe topic latency #893
- Monitoring API lacks topics and stateproof endpoint coverage #667
- Aggregate monitor API should return 4xx if individual server fails #462
Bug Fixes
- Fix NPE parsing token transactions #1157
- NullPointerException parsing token dissociate #1156
- Fix memory leak in performance tests #1072
- Re add deleted metrics #1058
- gRPC notify thread blocked improvement #1057
- Fix polling cloud storage more often than the configured poll period #1054
- Fix race condition in check-state-proof #1053
- Check-state-proof error #1052
Documentation
- Fix wrong command in check-state-proof readme #1055
Dependency Upgrades
- Bump docker-maven-plugin from 0.33.0 to 0.34.1 #1100
- Bump jacoco-maven-plugin from 0.8.5 to 0.8.6 #1099
- Bump testcontainers from 4.0.9 to 4.3.0 in /hedera-mirror-rest #1097
- Bump eslint-plugin-import from 2.22.0 to 2.22.1 in /hedera-mirror-rest #1096
- Bump sinon from 9.0.3 to 9.1.0 in /hedera-mirror-rest #1093
- Bump spring-cloud-dependencies from Hoxton.SR7 to Hoxton.SR8 #1084
- Bump eslint-config-prettier from 6.11.0 to 6.12.0 in /hedera-mirror-rest #1083
- Bump aws-sdk from 2.756.0 to 2.761.0 in /hedera-mirror-rest #1082
- Bump supertest from 4.0.2 to 5.0.0 in /hedera-mirror-rest #1081
- Bump mathjs from 7.2.0 to 7.3.0 in /hedera-mirror-rest #1080
- Bump prettier from 2.1.1 to 2.1.2 in /hedera-mirror-rest #1079
- Bump mathjs from 7.2.0 to 7.3.0 in /hedera-mirror-rest/monitoring/monitor_apis #1078
- Bump supertest from 4.0.2 to 5.0.0 in /hedera-mirror-rest/monitoring/monitor_apis #1077
- Bump cucumber.version from 6.6.0 to 6.7.0 #1066
- Bump sdk from 1.2.0 to 1.2.2 #1065
- Bump aws-sdk from 2.747.0 to 2.756.0 in /hedera-mirror-rest #1064
- Bump testcontainers from 3.4.4 to 4.0.9 in /hedera-mirror-rest #1063
- Bump eslint-plugin-jest from 24.0.0 to 24.0.2 in /hedera-mirror-rest #1062
- Bump embedded.testcontainers.version from 1.81 to 1.82 #1041
- Bump testcontainers from 3.4.3 to 3.4.4 in /hedera-mirror-rest #1039
- Bump pretty-quick from 3.0.0 to 3.0.2 in /hedera-mirror-rest #1036
- Bump husky from 4.2.5 to 4.3.0 in /hedera-mirror-rest #1035
- Bump protobuf-java from 3.11.1 to 3.13.0 #1024
- Bump reporting-plugin from 4.0.77 to 4.0.84 #1023
Contrib...
v0.21.0-rc2
Bump v0.21.0-rc2 with missing columns to token info REST API (#1173) * Bump v0.21.0-rc2 with missing columns to token info REST API - Add modified_timestamp and name to token info response - Update spec and its with missing columns - Bump from rc1 to rc2 Signed-off-by: Nana-EC <[email protected]>
v0.21.0-rc1
Bump version for v0.21.0-rc1 (#1161) Signed-off-by: Nana-EC <[email protected]>