Code Monkey home page Code Monkey logo

esb-inbound-dbevent's Introduction

esb-inbound-dbevent

The DB Event inbound endpoint allows you to capture the data change via WSO2 ESB.

##Build

mvn clean install

###How You Can Contribute

You can create a third party connector and publish in WSO2 Connector Store.

https://docs.wso2.com/display/ESBCONNECTORS/Creating+a+Third+Party+Connector+and+Publishing+in+WSO2+Connector+Store

Pre-requisites:

  • Maven 3.x
  • Java 1.6 or above
  • The JDBC driver e.g:mysql-connector-java-5.1.36-bin.jar.

Tested Platform:

  • Microsoft WINDOWS V-7
  • Mac OSX 10.11.6
  • Ubuntu 16.04 TLS
  • wso2ei-6.1.1
  • Java 1.8
  1. To use the DB Event inbound endpoint, you need to download the inbound org.apache.synapse.dbevent.poll-1.0.0.jar from https://store.wso2.com and copy the jar to the <EI_HOME>/lib directory.

  2. Place the mysql-connector-java-5.1.36-bin.jar into the directory <EI_HOME>/lib.

  3. Configuration:

<inboundEndpoint xmlns="http://ws.apache.org/ns/synapse"
                 name="customIE"
                 sequence="request"
                 onError="fault"
                 class="org.wso2.carbon.inbound.poll.dbeventlistener.DBEventPollingConsumer"
                 suspend="false">
   <parameters>
      <parameter name="inbound.behavior">polling</parameter>
      <parameter name="interval">10000</parameter>
      <parameter name="sequential">true</parameter>
      <parameter name="coordination">true</parameter>
      <parameter name="driverName">com.mysql.jdbc.Driver</parameter>
      <parameter name="url">jdbc:mysql://localhost/test</parameter>
      <parameter name="username">root</parameter>
      <parameter name="tableName">CDC_CUSTOM</parameter>
      <parameter name="filteringCriteria">byLastUpdatedTimestampColumn</parameter>
      <parameter name="filteringColumnName">LAST_UPDATED_DATE_TIME</parameter>
      <parameter name="primaryKey">ID</parameter>
      <parameter name="connectionValidationQuery">SELECT 1</parameter>
      <parameter name="registryPath">customIE/timestamp</parameter>
   </parameters>
</inboundEndpoint>
  1. driverName - The class name of the database driver.
  2. url - The JDBC URL of the database.
  3. username - The user name used to connect to the database.
  4. password - The password used to connect to the database.
  5. tableName - The name of the table to capture the change.
  6. filteringCriteria - It can be one of these byLastUpdatedTimestampColumn or byBooleanColumn or deleteAfterPoll.
  7. filteringColumnName - The actual name of table column. It must be set if the filteringCriteria has the value 'byLastUpdatedTimestampColumn' or 'byBooleanColumn'.
  8. primaryKey - The primary key column name.
  9. connectionValidationQuery - The query to check the availability of the connection.
  10. registryPath - The registry path of the timestamp. This will be used to retrieve the records when the filteringCriteria has the value 'byLastUpdatedTimestampColumn'.

esb-inbound-dbevent's People

Contributors

abeykoon avatar biruntha avatar connector-store-rw-bot avatar dilini-muthumala avatar janithcmw avatar keerthu avatar kesavany avatar maheshika avatar malakaganga avatar megala21 avatar menakajl avatar menuka94 avatar muthulee avatar nirthika avatar sachithkay avatar sdkottegoda avatar shakila avatar shayanmalinda avatar virajhasitha avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

esb-inbound-dbevent's Issues

DB Event Poller is storing an invalid value in registry for Oracle DB

Description:
Using an Oracle DB, the first trigger of the event poller appears to capture the update, and creates the registry record of the last update value. This value is being stored incorrectly:

¨Ì��p

Configuration:

<inboundEndpoint class="org.wso2.carbon.inbound.poll.dbeventlistener.DBEventPollingConsumer" name="watcher" onError="watcher.faulthandler" sequence="watcher.sequence” suspend="false" xmlns="http://ws.apache.org/ns/synapse">
    <parameters>
        <parameter name="inbound.behavior">polling</parameter>
        <parameter name="interval">60000</parameter>
        <parameter name="sequential">true</parameter>
        <parameter name="coordination">true</parameter>
        <parameter name="driverName">oracle.jdbc.driver.OracleDriver</parameter>
        <parameter name="url">jdbc:oracle:thin:******/******@******</parameter>
        <parameter name="tableName">TABLE_NAME</parameter>
        <parameter name="filteringCriteria">byLastUpdatedTimestampColumn</parameter>
        <parameter name="filteringColumnName">TO_CHAR(LAST_UPDATE_DATE, 'YYYY-MM-DD hh24:mm:ss')</parameter>
        <parameter name="primaryKey">ID</parameter>
        <parameter name="connectionValidationQuery">SELECT 1 FROM DUAL</parameter>
        <parameter name="registryPath">watcher/timestamp</parameter>
    </parameters>
</inboundEndpoint>

I've had to convert the filteringColumnName to the value shown because an error was thrown as Oracle can't compare dates as strings.

Every poll after the first (when the registry value has the incorrect value) triggers the following error repeatedly in the wso2carbon logs for EI:

[2019-10-11 19:01:36,406] [-1] [] [pool-29-thread-1] ERROR {org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter} -  Error in executing task: null
java.lang.NullPointerException
        at org.wso2.carbon.inbound.poll.dbeventlistener.DBEventPollingConsumer.fetchDataAndInject(DBEventPollingConsumer.java:182)
        at org.wso2.carbon.inbound.poll.dbeventlistener.DBEventPollingConsumer.poll(DBEventPollingConsumer.java:352)
        at org.wso2.carbon.inbound.endpoint.protocol.generic.GenericTask.taskExecute(GenericTask.java:41)
        at org.wso2.carbon.inbound.endpoint.common.InboundTask.execute(InboundTask.java:45)
        at org.wso2.carbon.mediation.ntask.NTaskAdapter.execute(NTaskAdapter.java:98)
        at org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter.execute(TaskQuartzJobAdapter.java:67)
        at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Suggested Labels:
bug

Affected Product Version:
WSO2 EI 6.5.0, most recent version of DB Event Listener (downloaded today) from the store:
https://store.wso2.com/store/assets/esbconnector/details/da8c396a-281e-493b-87bc-b55a4cebe92d

OS, DB, other environment details and versions:
DB: Oracle 12C
Environment: CentOS 7 VM

Steps to reproduce:
Use the configuration above to deploy on the specified environment, and once the poller is first triggered, check the wso2carbon logs or the registry value to find the error or the incorrect value.

Task to do when upgrading synapse-core

Description:
To fix the issue [2] in the EI 6.5.0 product, the property "inbound.endpoint.name" needs to be set to message context.
EI 6.5.0 packs synapse-core v2.1.7.wso2v111 and this property is defined in that synapse version. However the master branch of this repo currently depends on synapse-core v2.1.3-wso2v5 and this constant is not defined in this synapse version.
Therefore the constant is hard coded in this repo itself in [1].

Since defining the constant here might break things in future (in case the property name changes in future in the synapse-core), I have created this issue.

When upgrading the synapse-core dependency in this repo, please remove this constant definition in [1], and use the constant imported from synapse wherever it is used in this repo.

[1] https://github.com/wso2-extensions/esb-inbound-dbevent/blob/master/src/main/java/org/wso2/carbon/inbound/poll/dbeventlistener/DBEventConstants.java#L35
[2] wso2/product-ei#5527

Affected Product Version:
N/A
OS, DB, other environment details and versions:
N/A
Steps to reproduce:
N/A

Related Issues:
wso2/product-ei#5527

DB Event inbound not supporting CLOB columns

Description:
Cannot retrieve the CLOB column data in Oracle DB using the DB Event inbound.

Affected Product Version:
Enterprise Integrator

OS, DB, other environment details and versions:
Oracle DB

Steps to reproduce:
Use DB Event inbound to retrieve the data from a table where it contains a column with a CLOB data type gives the following error.

ERROR - DBEventPollingConsumer Unsupported column type 2005

DB Event Polling Inbound Endpoint Connector does not work with Oracle Databases

Description:
DB Event Polling Inbound Endpoint Connector does not work with Oracle Databases. The following error appears on connecting an Oracle database.

{org.wso2.carbon.inbound.poll.dbeventlistener.DBEventPollingConsumer}
- Error while checking the 
database connection.
java.sql.SQLException: Invalid operation for forward only resultset : first
at oracle.jdbc.driver.BaseResultSet.first(BaseResultSet.java:87)
at 
org.wso2.carbon.inbound.poll.dbeventlistener.DBEventPollingConsumer.isConnectionAlive(DBEventP
ollingConsumer.java:383)
at 
org.wso2.carbon.inbound.poll.dbeventlistener.DBEventPollingConsumer.poll(DBEventPollingConsume
r.java:343)

OS, DB, other environment details and versions:

Steps to reproduce:
Try out the docs [1][2] by using an Oracle database instead of MySQL.

[1] https://docs.wso2.com/display/ESBCONNECTORS/DB+Event+Listener
[2] https://docs.wso2.com/display/ESBCONNECTORS/Working+with+the+DB+Event+Listener

Related Issues:
#11

DB Event Polling inbound endpoint connector is not working with PostgreSQL DB

Description:
DB Event Polling inbound endpoint is not working for postgreSQL DB. When trying to connect it gives below error.
connection.org.postgresql.util.PSQLException: Can't use query methods that take a query string on a PreparedStatement

Steps to reproduce:

  1. Setup DB Event Polling inbound endpoint with a PostgreSQL DB
  2. When trying to start the IE it gives the error.

Error occured when we use this with H2 databse

Description:
org.h2.jdbc.JdbcSQLException: This method is not allowed for a prepared statement; use a regular statement instead. [90130-175]

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Micro Integrator 1.1 Deployment Error

ERROR {org.wso2.micro.integrator.ntask.core.impl.TaskQuartzJobAdapter} - Error in executing task: org/wso2/carbon/registry/api/RegistryException java.lang.NoClassDefFoundError: org/wso2/carbon/registry/api/RegistryException
at org.wso2.carbon.inbound.poll.dbeventlistener.DBEventPollingConsumer.fetchDataAndInject(DBEventPollingConsumer.java:179)
at org.wso2.carbon.inbound.poll.dbeventlistener.DBEventPollingConsumer.poll(DBEventPollingConsumer.java:352)
at org.wso2.carbon.inbound.endpoint.protocol.generic.GenericTask.taskExecute(GenericTask.java:40)
at org.wso2.carbon.inbound.endpoint.common.InboundTask.execute(InboundTask.java:43)
at org.wso2.micro.integrator.mediation.ntask.NTaskAdapter.execute(NTaskAdapter.java:105)
at org.wso2.micro.integrator.ntask.core.impl.TaskQuartzJobAdapter.execute(TaskQuartzJobAdapter.java:63)
at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.wso2.carbon.registry.api.RegistryException cannot be found by org.apache.synapse.poll.dbeventlistener_1.0.3_1.0.0
at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:512)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:423)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:415)
at org.eclipse.osgi.internal.loader.ModuleClassLoader.loadClass(ModuleClassLoader.java:155)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 12 more

[2020-01-09 12:30:20,331] ERROR {org.apache.synapse.transport.nhttp.access} - Error occurred when parsing the date from existing log file java.text.ParseException: Unparseable date: "09/Jan/2020:12:22:14 -0300"
at java.text.DateFormat.parse(DateFormat.java:366)
at org.apache.synapse.transport.http.access.AccessLogger.open(AccessLogger.java:256)
at org.apache.synapse.transport.http.access.AccessLogger.initOpen(AccessLogger.java:212)
at org.apache.synapse.transport.http.access.AccessLogger.(AccessLogger.java:66)
at org.apache.synapse.transport.http.access.AccessHandler.(AccessHandler.java:32)
at org.apache.synapse.transport.http.conn.LoggingNHttpServerConnection$LoggingNHttpMessageParser.parse(LoggingNHttpServerConnection.java:430)
at org.apache.synapse.transport.http.conn.LoggingNHttpServerConnection$LoggingNHttpMessageParser.parse(LoggingNHttpServerConnection.java:381)
at org.apache.http.impl.nio.DefaultNHttpServerConnection.consumeInput(DefaultNHttpServerConnection.java:265)
at org.apache.synapse.transport.http.conn.LoggingNHttpServerConnection.consumeInput(LoggingNHttpServerConnection.java:114)
at org.apache.synapse.transport.passthru.core.MultiListenerServerIODispatch.onInputReady(MultiListenerServerIODispatch.java:83)
at org.apache.synapse.transport.passthru.core.MultiListenerServerIODispatch.onInputReady(MultiListenerServerIODispatch.java:32)
at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:113)
at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:159)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:338)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:316)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:277)
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:105)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:586)
at java.lang.Thread.run(Thread.java:748)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.