Releases: hashgraph/hedera-mirror-node
v0.33.0-rc2
1c7c054v0.33.0-rc2
v0.33.0-rc1
c78ff0fv0.33.0-rc1
v0.33.0-alpha1
Bump versions for v0.33.0-alpha1 (#1901) Signed-off-by: Steven Sheehy <[email protected]>
v0.32.0
This release we took the time to do some performance optimizations of both the importer and the monitor. If you're using a containerized mirror node, the Java applications now uses more of the available memory that's already been allocated to it. We optimized the size of some internal queues to reduce the likelihood of out of memory errors. And we now use a more efficient streaming method to write entities to the database to avoid large memory allocations. All these combine to greatly reduce memory usage and improve overall performance for the importer. The monitor also saw performance improvements to allow it to publish transactions at a rate of 10,000 TPS.
This release updates more of our system to handle the revised scheduled transaction design that will be available soon on mainnet. Both the acceptance tests and monitor were updated to be able to publish the new transactions.
We now expose the raw transaction bytes encoded in Base64 format in the REST API. Persisting the bytes of the Transaction protobuf in the database is an option that's been available for awhile but until now has not been available via the API. Persisting the data is off by default as does increase the size of the database quite a bit. The Hedera managed mirror nodes will not have that functionality turned on to reduce storage.
Enhancements
- Bump versions for v0.32.0 #1902
- Bump versions for v0.32.0-rc1 #1867
- Optimize resources for kubernetes deployment #1853
- Add cloud storage metrics #1848
- Updated scheduled tests #1846
- Expose transaction bytes in REST API #1823
- Update scheduled transaction acceptance tests #1822
- Updated scheduled transactions support in monitor #1821
- Expose transaction bytes via REST API #1812
Bug Fixes
- Fix deadlock due to not cleaning up PostgreSQL CopyIn when error #1894
- Record parsing deadlock #1891
- vacuum error in k8s with timescaledb #1849
- Importer in GKE OOM in catch-up mode #1830
- Fix dashboards and alerts #1827
- fix test failure caused by timing precision #1826
- Autovacuum crypto_transfer, token_transfer, and transaction #1825
- CompositeTransactionGeneratorTest.scenariosDurationAfterBothFinish fails #1819
- Improve monitor performance #1816
- Monitor with SDK v2 performs much worse under high TPS #1814
- Querying REST api accounts endpoint for 0.0.9 times out in production environment #1780
Dependency Upgrades
- Bump js-yaml from 4.0.0 to 4.1.0 in /hedera-mirror-rest #1861
- Bump eslint-plugin-prettier from 3.3.1 to 3.4.0 in /hedera-mirror-rest #1860
- Bump pg from 8.5.1 to 8.6.0 in /hedera-mirror-rest #1858
- Bump aws-sdk from 2.879.0 to 2.888.0 in /hedera-mirror-rest #1857
- Bump mathjs from 9.3.0 to 9.3.2 in /hedera-mirror-rest/monitoring/monitor_apis #1856
- Bump mathjs from 9.3.0 to 9.3.2 in /hedera-mirror-rest #1855
- Bump boxen from 5.0.0 to 5.0.1 in /hedera-mirror-rest/check-state-proof #1841
- Bump grpc.version from 1.36.1 to 1.37.0 #1839
- Bump protobuf-java from 3.15.7 to 3.15.8 #1838
- Bump eslint-plugin-jest from 24.3.4 to 24.3.5 in /hedera-mirror-rest #1835
- Bump
@hashgraph/proto from 1.0.23 to 1.0.24 in /hedera-mirror-rest #1834 - Bump husky from 5.2.0 to 6.0.0 in /hedera-mirror-rest #1799
Contributors
We'd like to thank all the contributors who worked on this release!
v0.32.0-rc2
Fix deadlock due to not cleaning up PostgreSQL CopyIn when error (#1894) * Fix deadlock due to not cleaning up PostgreSQL CopyIn when encountering an error * Bump versions for v0.32.0-rc2 * Disable schedule crypto create in acceptance tests Signed-off-by: Steven Sheehy <[email protected]>
v0.32.0-rc1
Bump versions for v0.32.0-rc1 (#1867) * Bump versions for v0.32.0-rc1 Signed-off-by: Steven Sheehy <[email protected]>
v0.31.0
After scheduled transactions was made available in previewnet, we listened to user feedback and further iterated on the design to make it easier to use. This release adds support for this revised scheduled transactions design planned to be released in HAPI v0.13. There was no impact to our REST API format, only the importer needed to be updated to parse and ingest the new proto format. Our monitor API and acceptance tests will be adjusted in the next release once the SDKs add support for the new design.
This release also adds support for the newly announced account balance file format that was released in HAPI v0.12. The new protobuf based format will eventually replace the CSV format in July 2021. Until then, both formats will exist simultaneously in the bucket so users can transition at their leisure. Besides being more efficient to parse, the new files are also compressed using Gzip for reduced storage and download costs. We also took the time to improve the balance file parsing performance regardless of format. Average parse times should decrease by about 27%.
For our REST API, we now expose an entity_id field on our transactions related APIs. This field represents that main entity associated with that transaction type. For example, if it was a HCS transaction it would be the topic ID created, updated, or deleted.
GET /api/v1/transactions/0.0.1009-1234567890-999999998
{
"transactions": [{
"consensus_timestamp": "1234567890.999999999",
"entity_id": "0.0.108763",
"valid_start_timestamp": "1234567890.999999998",
"charged_tx_fee": 0,
"memo_base64": null,
"result": "SUCCESS",
"scheduled": false,
"transaction_hash": "aGFzaA==",
"name": "CRYPTOUPDATEACCOUNT",
"node": "0.0.3",
"transaction_id": "0.0.1009-1234567890-999999998",
"valid_duration_seconds": "11",
"max_fee": "33",
"transfers": []
}]
}We continue to make progress towards our goal of switching to TimescaleDB. We fixed the user and database initialization issues and tested a migration from PostgreSQL. We switched out the TimescaleDB Helm chart to a more stable one and explored our hosting options for production. Finally we switched to SCRAM-SHA-256 to improve the security of our database user authentication.
Breaking changes:
There were a number of breaking changes this release to be aware of. If you're using our Helm chart, we have switched the importer from a StatefulSet to a Deployment since it no longer has the need for a persistent volume. We also switched the Traefik dependency from a Deployment to a DaemonSet. Both of these will require manual intervention to delete the old workload before upgrading. You'll also need to delete the old leader election configmap since it will still references the importer StatefulSet (e.g. kubectl delete configmap leaders). Support for Helm 2 was dropped since it is no longer supported by the community after November 13, 2020. If you're directly reading from our database, a rename of the t_entities table and its columns may impact you as well.
Enhancements
- Bump versions for v0.31.0 #1843
- Bump versions for v0.31.0-rc1 #1818
- Update entity table references in entity migration tool #1815
- Fixes to the timescale migration script #1798
- Change importer to Deployment & bump dependencies #1797
- Use scram-sha-256 password authentication for TimescaleDB #1795
- Expose entity_id in transactions REST API #1791
- Move Sonar to separate job & fix chart test #1768
- Update importer for revised scheduled transaction design #1766
- Add postInit to timescaledb chart #1764
- Helm v3 #1752
- Use secure authentication for TimescaleDB #1739
- Update REST API for revised scheduled transaction design #1738
- Update importer for revised scheduled transaction design #1737
- Proto based balance file parser #1713
- Parse proto-based balance file #1679
- Expose entity_id on /api/v1/transactions endpoint #1639
- Upgrade to Helm 3 #1502
- Properly initialize timescaledb before other resources access it #1439
- TimescaleDB: Test data migration from pre-prod environment to k8s cluster #1423
- TimescaleDB: Explore Hosting Options #1422
Bug Fixes
- Fix REST API image #1817
- Port multi-scenario running longer fix #1807
- Fix test image build #1794
- Fix master deploy failing #1793
- Fix REST artifact missing v prefix #1787
- Fix jar name & bump to v0.31.0-alpha3 #1786
- Fix release artifact & bump to v0.31.0-alpha2 #1784
- Migrate and update t_entities to entity table #1779
- Monitor may run a scenario for multiple of the configured duration / number of transactions #1777
- Improve balance file performance #1767
- Clean up t_entities columns with empty values #1740
- Balance file parsing performance #1736
- Provide migration tool to fix incorrect keys in t_entities #1721
- Incorrect entity keys #509
Dependency Upgrades
- Bump docker-maven-plugin from 0.34.1 to 0.35.0 #1810
- Bump download-maven-plugin from 1.6.1 to 1.6.2 #1805
- Bump software.amazon.awssdk:bom from 2.16.31 to 2.16.34 #1804
- Bump protobuf-java from 3.15.6 to 3.15.7 #1803
- Bump aws-sdk from 2.876.0 to 2.879.0 in /hedera-mirror-rest #1801
- Bump eslint-plugin-jest from 24.3.2 to 24.3.4 in /hedera-mirror-rest #1800
- Bump dependency-check-maven from 6.1.2 to 6.1.5 #1792
- Bump aws-sdk from 2.871.0 to 2.876.0 in /hedera-mirror-rest #1790
- Bump y18n from 4.0.0 to 4.0.1 in /hedera-mirror-rest/monitoring/monitor_apis #1789
- Bump y18n from 4.0.0 to 4.0.1 in /hedera-mirror-rest #1788
- Bump reporting-plugin from 4.0.98 to 4.0.101 #1782
- Bump software.amazon.awssdk:bom from 2.16.26 to 2.16.31 #1781
- Bump
@hashgraph/proto from 1.0.22 to 1.0.23 in /hedera-mirror-rest/check-state-proof #1776 - Bump frontend-maven-plugin from 1.11.2 to 1.11.3 #1773
- Bump testcontainers from 7.6.2 to 7.7.0 in /hedera-mirror-rest #1771
- Bump
@hashgraph/proto from 1.0.22 to 1.0.23 in /hedera-mirror-rest #1769 - Bump spring-boot-starter-parent from 2.4.3 to 2.4.4 #1765
- Bump grpc-bom from 1.36.0 to 1.36.1 #1763
- Bump mvn-golang-wrapper from 2.3.7 to 2.3.8 #1762
- Bump dependency-check-maven from 6.1.2 to 6.1.3 #1761
- Bump embedded.testcontainers.version from 2.0.4 to 2.0.5 [#1760](https://github.com/hashgraph/hedera-mirror-no...
v0.31.0-rc1
Bump versions for v0.31.0-rc1 (#1818) * Bump versions for v0.31.0-rc1 * Fix TimescaleDB chart test failing on version bumps Signed-off-by: Steven Sheehy <[email protected]>
v0.30.0
Mirror node v0.30 brings operational improvements with changes to our continuous integration and monitoring components.
With this release, we've completed the migration from CircleCI to GitHub Actions. CircleCI had some limitations with our use of Testcontainers for unit testing against 3rd party dependencies. We previously had a mixture of GitHub Actions and CircleCI with the latter using slightly different commands than local testing. Consolidating to GitHub Actions allowed us to reduce this difference and further parallelize our checks.
To improve our runtime observability and testing coverage, we've continued to invest in our monitor tool this cycle. Scheduled transaction support was recently added supporting both ScheduleCreate and ScheduleSign operations. We've added the three new mainnet nodes the monitor's default configuration. A bug with the monitor unable to reach expected TPS with multiple scenarios was fixed.
The REST API also saw some bug fixes including a fix to queries with a credit/debit parameter now able to retrieve token only transfers. The transaction API now populates the token transfers list for all transaction types instead of being limited to just crypto transfers.
Enhancements
- Bump versions for v0.30.0 #1808
- Migrate CircleCI to GitHub Actions #1723
- Add new mainnet nodes to monitor #1719
- Add dependency-check-maven plugin to catch vulnerable dependencies #1701
- Add schedule transactions support to monitor #1688
- Add Rosetta to SonarCloud #1589
- Scheduled transactions monitor #1469
- Migrate from CircleCI to GitHub Actions #1112
Bug Fixes
- Fix release artifact & bump version to v0.30.0-rc2 #1785
- Fix accounts transaction REST API timeouts (master) #1751
- Fix charts workflow and ConsensusControllerTest #1747
- Fix HistoricalAccountInfoMigration unable to find accountInfo file (master) #1745
- Fix Monitor can't reach expected tps #1724
- Fix npm run lint #1718
- Replace null character on insert #1716
- Address audit static analysis problems #1715
- Add ability for REST transaction queries with credit/debit parameter to retrieve token only transfers #1714
- git ignore check-state-proof output files #1704
- Correct REST transaction endpoints to include token_transfer_list on token wipes #1703
- Querying Credit Type Transfers for a recipient account does not retrieve token transfers #1686
- RecordFileParserPerformanceTest times out in CircleCi #1647
- RedisEntityListenerIntegrationTest.onTopicMessage fails in CircleCI #1646
- Monitor unable to reach expected TPS with multiple scenarios #1606
Dependency Upgrades
- Bump
@hashgraph/proto from 1.0.20 to 1.0.21 in /hedera-mirror-rest/check-state-proof #1712 - Bump vertx-pg-client from 4.0.2 to 4.0.3 #1711
- Bump protobuf-java from 3.15.5 to 3.15.6 #1710
- Bump software.amazon.awssdk:bom from 2.16.15 to 2.16.18 #1709
- Bump aws-sdk from 2.860.0 to 2.863.0 in /hedera-mirror-rest #1707
- Bump
@hashgraph/proto from 1.0.20 to 1.0.21 in /hedera-mirror-rest #1706 - Bump eslint-plugin-jest from 24.2.1 to 24.3.1 in /hedera-mirror-rest #1705
- Bump mathjs from 9.2.0 to 9.3.0 in /hedera-mirror-rest #1698
- Bump aws-sdk from 2.858.0 to 2.860.0 in /hedera-mirror-rest #1697
- Bump testcontainers from 7.4.0 to 7.5.0 in /hedera-mirror-rest #1696
- Bump eslint-plugin-jest from 24.1.8 to 24.2.1 in /hedera-mirror-rest #1695
- Bump boxen from 4.2.0 to 5.0.0 in /hedera-mirror-rest/check-state-proof #1694
- Bump lodash from 4.17.19 to 4.17.21 in /hedera-mirror-rest/check-state-proof #1693
- Bump jest from 26.4.0 to 26.6.3 in /hedera-mirror-rest/check-state-proof #1692
- Bump mathjs from 9.2.0 to 9.3.0 in /hedera-mirror-rest/monitoring/monitor_apis #1691
- Bump cucumber.version from 6.10.0 to 6.10.1 #1690
- Bump software.amazon.awssdk:bom from 2.16.13 to 2.16.15 #1689
Contributors
We'd like to thank all the contributors who worked on this release!
v0.30.0-rc2
Fix release artifact & bump version to v0.30.0-rc2 (#1785) - Fix Rosetta validation failing on tags - Fix release jar not prefixed with `v` - Fix release tgz not prefixed with `v` - Bump version to v0.30.0-rc2 Signed-off-by: Steven Sheehy <[email protected]>