[Bug] [Seatunnel-web] HiveJDBC should not be used as a sink datasource

[arshadmohammad]

Removed HDBC-Hive from sink datasource list

Purpose of this pull request

Check list

• Code changed are covered with tests, or it does not need tests for reason:
• If any new Jar binary package adding in your PR, please add License Notice according
  New License Guide
• If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs

[arshadmohammad]

After the fix, JDBC-Hive datasource is not available in sink connector configurations

Even though hive-jdbc-1 and hive-jdbc-2 are available in the system, they are not available for selection in sink connector.

[dragongu]

@arshadmohammad

Hello, may I ask if the current version supports Elasticsearch as a sink?
When I create an Elasticsearch sink, it always requires me to enter a fixed and pre-existing table name.

[arshadmohammad]

Hi @dragongu, Seatunnel engine and Seatunnel-web both support the Elasticsearch sink.
Please verify the functionality to ensure it is working correctly. If you identify any issues, kindly log a bug report and, if possible, provide a fix.

[arshadmohammad]

@Hisoka-X can you review and merge this and other PRs

[avatarTaiers]

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2024-10-04 21:52:42.400 seatunnel iZuf6bqkrgecwbiaz7wam7Z ERROR [main] [SpringApplication.reportFailure():835] - Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'connectorCache' defined in URL [jar:file:/seatunnel/apache-seatunnel-web-1.0.1/libs/seatunnel-app-1.0.1.jar!/org/apache/seatunnel/app/bean/connector/ConnectorCache.class]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.apache.seatunnel.app.bean.connector.ConnectorCache]: Constructor threw exception; nested exception is java.lang.NoSuchFieldError: APPEND_DATA
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1334)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1232)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:953)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:145)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:745)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:420)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:307)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1317)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306)
at org.apache.seatunnel.app.SeatunnelApplication.main(SeatunnelApplication.java:36)
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.apache.seatunnel.app.bean.connector.ConnectorCache]: Constructor threw exception; nested exception is java.lang.NoSuchFieldError: APPEND_DATA
at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:224)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:87)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:1326)
... 17 common frames omitted
Caused by: java.lang.NoSuchFieldError: APPEND_DATA
at org.apache.seatunnel.connectors.seatunnel.paimon.config.PaimonSinkConfig.(PaimonSinkConfig.java:42)
at org.apache.seatunnel.connectors.seatunnel.paimon.sink.PaimonSinkFactory.optionRule(PaimonSinkFactory.java:48)
at org.apache.seatunnel.api.table.factory.FactoryUtil.sinkFullOptionRule(FactoryUtil.java:282)
at org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery.lambda$getAllPlugin$8(AbstractPluginDiscovery.java:316)
at java.base/java.lang.Iterable.forEach(Iterable.java:75)
at org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery.getAllPlugin(AbstractPluginDiscovery.java:291)
at org.apache.seatunnel.app.thirdparty.framework.PluginDiscoveryUtil.getAllConnectors(PluginDiscoveryUtil.java:126)
at org.apache.seatunnel.app.bean.connector.ConnectorCache.refresh(ConnectorCache.java:107)
at org.apache.seatunnel.app.bean.connector.ConnectorCache.(ConnectorCache.java:73)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:211)
... 19 common frames omitted

[dragongu]

Hi @dragongu, Seatunnel engine and Seatunnel-web both support the Elasticsearch sink. Please verify the functionality to ensure it is working correctly. If you identify any issues, kindly log a bug report and, if possible, provide a fix.

@arshadmohammad
When using seatunnel-web, I expect that when an index does not exist, it will be automatically created, such as indexprefix_${day}. However, when configuring the es sink, seatunnel-web requires me to select a specific index. The seatunnel engine supports this feature.