Code Monkey home page Code Monkey logo

tdrules's People

Contributors

claudiodelariva avatar dependabot[bot] avatar giis-qabot avatar javiertuya avatar

Stargazers

avatar avatar avatar avatar

Watchers

avatar avatar

tdrules's Issues

Fix breaking changes openapi generator 7 for .NET

Update to generator 7.0.0 in .NET raises breaking changes that need to be addressed (requirement of C# v8, nullable handling).
Some references to address the issues:

Workaround: lock .net into openapi v6.6.0

Review readme

Prepare a PR with a revised version of the README.md according to our previous discussion

Migrate model sorting and filtering

Move the classes to sort and filter that depend only on the model from the on premises repo to tdrules-model. Deprecate old classes, but keep the sorting in tdrules-store-rdb

Matching of specified string values with dictionary values

Given a string attribute that has a dictionary, and a specified value to load:

  • If this value matches any in the dictionary and has a string format, it should be removed from the dictionary.
  • If this value does not match any in the dictionary and it has a numeric format, it sould get a value from the dictionary and remember matching the value with the dictionary-supplied for any further load.

Examples: dictionary contains "Tiger", "Lion", "Monkey",

  • the specified value is "Lion": this value will be removed from dictionary to prevent its use in further load calls that do not specify any value.
  • the specified value is 1: the generated value will be "Tiger" whenever 1 is specified, and "Tiger" removed from the dictionary

Update to httpclient5 v5.2.2 fails with an invalid proxy exception

If this new version of httpclient5 is used, calls made from the client fail with this exception:

java.lang.IllegalArgumentException: Invalid Proxy
at java.base/java.net.Socket.<init>(Unknown Source)
at org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory.createSocket(SSLConnectionSocketFactory.java:208)
at org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:158)
at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:447)
at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:162)
at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:172)
at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:142)
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96)
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
at org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:115)
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:106)
at giis.tdrules.openapi.invoker.ApiClient.invokeAPI(ApiClient.java:953)
at giis.tdrules.openapi.api.TdRulesApi.rulesPost(TdRulesApi.java:400)
at giis.tdrules.openapi.api.TdRulesApi.rulesPost(TdRulesApi.java:351)
at giis.tdrules.client.TdRulesApi.getRules(TdRulesApi.java:55)

Maybe it depends on how the openapi generator creates the api client (default version of httpclient5 is still 5.1.3. Worked until now with 5.2.1

Workaround: delete the update branch to close PR that udpates to this version.

Exception reading metadata of Cassandra using cassandra-jdbc-wrapper driver from com.ing.data

I'm trying to change Cassandra JDBC driver in QAGrow tool.
Until now, micromux driver was used but it has dependency conflicts and it is not available in Maven Central.

In Maven Central, there is cassandra-jdbc-wrapper from com.ing.data. However, when reading metadata (fks) from Cassandra database, giis.tdrules.store.rdb.SchemaReaderJdbc, using this driver, throws the exception:

giis.tdrules.store.rdb.SchemaReaderJdbc  - SchemaReaderJdbc.getTableList: 
java.sql.SQLFeatureNotSupportedException: The Cassandra implementation does not support this method.
	at com.ing.data.cassandra.jdbc.CassandraDatabaseMetaData.getImportedKeys(CassandraDatabaseMetaData.java:392)

Log information executing test case testTargetTables in class testUnitCassandra in QAGrow tool:

16:48:32,627 [main] INFO  in2test.util.Log  - **** Test case: testTargetTables
16:48:33,095 [main] DEBUG io.netty.util.internal.logging.InternalLoggerFactory  - Using SLF4J as the default logging framework
16:48:33,095 [main] DEBUG io.netty.util.internal.InternalThreadLocalMap  - -Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024
16:48:33,096 [main] DEBUG io.netty.util.internal.InternalThreadLocalMap  - -Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096
16:48:33,196 [main] INFO  com.datastax.oss.driver.internal.core.DefaultMavenCoordinates  - DataStax Java driver for Apache Cassandra(R) (com.datastax.oss:java-driver-core) version 4.17.0
16:48:33,207 [main] DEBUG io.netty.channel.MultithreadEventLoopGroup  - -Dio.netty.eventLoopThreads: 16
16:48:33,218 [main] DEBUG io.netty.util.concurrent.GlobalEventExecutor  - -Dio.netty.globalEventExecutor.quietPeriodSeconds: 1
16:48:33,247 [main] DEBUG io.netty.util.internal.PlatformDependent0  - -Dio.netty.noUnsafe: false
16:48:33,248 [main] DEBUG io.netty.util.internal.PlatformDependent0  - Java version: 19
16:48:33,248 [main] DEBUG io.netty.util.internal.PlatformDependent0  - sun.misc.Unsafe.theUnsafe: available
16:48:33,249 [main] DEBUG io.netty.util.internal.PlatformDependent0  - sun.misc.Unsafe.copyMemory: available
16:48:33,250 [main] DEBUG io.netty.util.internal.PlatformDependent0  - sun.misc.Unsafe.storeFence: available
16:48:33,252 [main] DEBUG io.netty.util.internal.PlatformDependent0  - java.nio.Buffer.address: available
16:48:33,253 [main] DEBUG io.netty.util.internal.PlatformDependent0  - direct buffer constructor: unavailable: Reflective setAccessible(true) disabled
16:48:33,255 [main] DEBUG io.netty.util.internal.PlatformDependent0  - java.nio.Bits.unaligned: available, true
16:48:33,257 [main] DEBUG io.netty.util.internal.PlatformDependent0  - jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable: class io.netty.util.internal.PlatformDependent0$7 cannot access class jdk.internal.misc.Unsafe (in module java.base) because module java.base does not export jdk.internal.misc to unnamed module @725bef66
16:48:33,259 [main] DEBUG io.netty.util.internal.PlatformDependent0  - java.nio.DirectByteBuffer.<init>(long, {int,long}): unavailable
16:48:33,260 [main] DEBUG io.netty.util.internal.PlatformDependent  - sun.misc.Unsafe: available
16:48:33,285 [main] DEBUG io.netty.util.internal.PlatformDependent  - maxDirectMemory: 4188012544 bytes (maybe)
16:48:33,286 [main] DEBUG io.netty.util.internal.PlatformDependent  - -Dio.netty.tmpdir: C:\Users\MARAJO~1\AppData\Local\Temp (java.io.tmpdir)
16:48:33,286 [main] DEBUG io.netty.util.internal.PlatformDependent  - -Dio.netty.bitMode: 64 (sun.arch.data.model)
16:48:33,287 [main] DEBUG io.netty.util.internal.PlatformDependent  - Platform: Windows
16:48:33,289 [main] DEBUG io.netty.util.internal.PlatformDependent  - -Dio.netty.maxDirectMemory: -1 bytes
16:48:33,290 [main] DEBUG io.netty.util.internal.PlatformDependent  - -Dio.netty.uninitializedArrayAllocationThreshold: -1
16:48:33,292 [main] DEBUG io.netty.util.internal.CleanerJava9  - java.nio.ByteBuffer.cleaner(): available
16:48:33,292 [main] DEBUG io.netty.util.internal.PlatformDependent  - -Dio.netty.noPreferDirect: false
16:48:33,293 [main] DEBUG io.netty.channel.nio.NioEventLoop  - -Dio.netty.noKeySetOptimization: false
16:48:33,293 [main] DEBUG io.netty.channel.nio.NioEventLoop  - -Dio.netty.selectorAutoRebuildThreshold: 512
16:48:33,301 [main] DEBUG io.netty.util.internal.PlatformDependent  - org.jctools-core.MpscChunkedArrayQueue: available
16:48:33,420 [main] DEBUG io.netty.util.ResourceLeakDetector  - -Dio.netty.leakDetection.level: simple
16:48:33,420 [main] DEBUG io.netty.util.ResourceLeakDetector  - -Dio.netty.leakDetection.targetRecords: 4
16:48:33,427 [main] DEBUG io.netty.util.ResourceLeakDetectorFactory  - Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@62727399
16:48:33,473 [main] INFO  com.datastax.oss.driver.internal.core.cql.CqlPrepareAsyncProcessor  - Adding handler to invalidate cached prepared statements on type changes
16:48:33,765 [s0-admin-0] INFO  com.datastax.oss.driver.internal.core.time.Clock  - Using native clock for microsecond precision
16:48:33,810 [s0-admin-1] DEBUG io.netty.buffer.PooledByteBufAllocator  - -Dio.netty.allocator.numHeapArenas: 16
16:48:33,810 [s0-admin-1] DEBUG io.netty.buffer.PooledByteBufAllocator  - -Dio.netty.allocator.numDirectArenas: 16
16:48:33,810 [s0-admin-1] DEBUG io.netty.buffer.PooledByteBufAllocator  - -Dio.netty.allocator.pageSize: 8192
16:48:33,810 [s0-admin-1] DEBUG io.netty.buffer.PooledByteBufAllocator  - -Dio.netty.allocator.maxOrder: 9
16:48:33,810 [s0-admin-1] DEBUG io.netty.buffer.PooledByteBufAllocator  - -Dio.netty.allocator.chunkSize: 4194304
16:48:33,811 [s0-admin-1] DEBUG io.netty.buffer.PooledByteBufAllocator  - -Dio.netty.allocator.smallCacheSize: 256
16:48:33,811 [s0-admin-1] DEBUG io.netty.buffer.PooledByteBufAllocator  - -Dio.netty.allocator.normalCacheSize: 64
16:48:33,811 [s0-admin-1] DEBUG io.netty.buffer.PooledByteBufAllocator  - -Dio.netty.allocator.maxCachedBufferCapacity: 32768
16:48:33,811 [s0-admin-1] DEBUG io.netty.buffer.PooledByteBufAllocator  - -Dio.netty.allocator.cacheTrimInterval: 8192
16:48:33,811 [s0-admin-1] DEBUG io.netty.buffer.PooledByteBufAllocator  - -Dio.netty.allocator.cacheTrimIntervalMillis: 0
16:48:33,811 [s0-admin-1] DEBUG io.netty.buffer.PooledByteBufAllocator  - -Dio.netty.allocator.useCacheForAllThreads: false
16:48:33,811 [s0-admin-1] DEBUG io.netty.buffer.PooledByteBufAllocator  - -Dio.netty.allocator.maxCachedByteBuffersPerChunk: 1023
16:48:33,820 [s0-admin-1] DEBUG io.netty.buffer.ByteBufUtil  - -Dio.netty.allocator.type: pooled
16:48:33,820 [s0-admin-1] DEBUG io.netty.buffer.ByteBufUtil  - -Dio.netty.threadLocalDirectBufferSize: 0
16:48:33,821 [s0-admin-1] DEBUG io.netty.buffer.ByteBufUtil  - -Dio.netty.maxThreadLocalCharBufferSize: 16384
16:48:33,840 [s0-admin-1] DEBUG io.netty.channel.DefaultChannelId  - -Dio.netty.processId: 2232 (auto-detected)
16:48:33,842 [s0-admin-1] DEBUG io.netty.util.NetUtil  - -Djava.net.preferIPv4Stack: false
16:48:33,843 [s0-admin-1] DEBUG io.netty.util.NetUtil  - -Djava.net.preferIPv6Addresses: false
16:48:33,854 [s0-admin-1] DEBUG io.netty.util.NetUtilInitializations  - Loopback interface: lo (Software Loopback Interface 1, 127.0.0.1)
16:48:33,855 [s0-admin-1] DEBUG io.netty.util.NetUtil  - Failed to get SOMAXCONN from sysctl and file \proc\sys\net\core\somaxconn. Default: 200
16:48:33,880 [s0-admin-1] DEBUG io.netty.channel.DefaultChannelId  - -Dio.netty.machineId: 58:96:1d:ff:fe:a2:44:1e (auto-detected)
16:48:33,967 [s0-io-0] DEBUG io.netty.util.Recycler  - -Dio.netty.recycler.maxCapacityPerThread: 4096
16:48:33,967 [s0-io-0] DEBUG io.netty.util.Recycler  - -Dio.netty.recycler.ratio: 8
16:48:33,967 [s0-io-0] DEBUG io.netty.util.Recycler  - -Dio.netty.recycler.chunkSize: 32
16:48:33,967 [s0-io-0] DEBUG io.netty.util.Recycler  - -Dio.netty.recycler.blocking: false
16:48:33,967 [s0-io-0] DEBUG io.netty.util.Recycler  - -Dio.netty.recycler.batchFastThreadLocalOnly: true
16:48:33,975 [s0-io-0] DEBUG io.netty.buffer.AbstractByteBuf  - -Dio.netty.buffer.checkAccessible: true
16:48:33,975 [s0-io-0] DEBUG io.netty.buffer.AbstractByteBuf  - -Dio.netty.buffer.checkBounds: true
16:48:33,975 [s0-io-0] DEBUG io.netty.util.ResourceLeakDetectorFactory  - Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@65e2aafc
16:48:34,710 [main] INFO  com.ing.data.cassandra.jdbc.CassandraConnection  - Connected to cluster: Test Cluster, with session: s0
16:48:34,711 [main] INFO  com.ing.data.cassandra.jdbc.CassandraConnection  - Datacenter: datacenter1; Host: localhost/<unresolved>:9042; Rack: rack1
16:48:34,712 [main] INFO  com.ing.data.cassandra.jdbc.CassandraConnection  - Node: localhost/<unresolved>:9042 runs Cassandra v.4.0.4
16:48:45,395 [s0-io-0] DEBUG io.netty.buffer.PoolThreadCache  - Freed 2 thread-local buffer(s) from thread: s0-io-0
16:48:45,395 [s0-io-1] DEBUG io.netty.buffer.PoolThreadCache  - Freed 2 thread-local buffer(s) from thread: s0-io-1
16:48:47,437 [s0-io-2] DEBUG io.netty.buffer.PoolThreadCache  - Freed 15 thread-local buffer(s) from thread: s0-io-2
16:48:47,437 [s0-io-3] DEBUG io.netty.buffer.PoolThreadCache  - Freed 14 thread-local buffer(s) from thread: s0-io-3
16:48:47,508 [main] INFO  com.datastax.oss.driver.internal.core.cql.CqlPrepareAsyncProcessor  - Adding handler to invalidate cached prepared statements on type changes
16:48:47,510 [s1-admin-0] INFO  com.datastax.oss.driver.internal.core.time.Clock  - Using native clock for microsecond precision
16:48:47,749 [main] INFO  com.ing.data.cassandra.jdbc.CassandraConnection  - Connected to cluster: Test Cluster, with session: s1
16:48:47,750 [main] INFO  com.ing.data.cassandra.jdbc.CassandraConnection  - Datacenter: datacenter1; Host: localhost/<unresolved>:9042; Rack: rack1
16:48:47,750 [main] INFO  com.ing.data.cassandra.jdbc.CassandraConnection  - Node: localhost/<unresolved>:9042 runs Cassandra v.4.0.4
16:49:00,872 [s1-io-1] DEBUG io.netty.buffer.PoolThreadCache  - Freed 2 thread-local buffer(s) from thread: s1-io-1
16:49:00,872 [s1-io-0] DEBUG io.netty.buffer.PoolThreadCache  - Freed 2 thread-local buffer(s) from thread: s1-io-0
16:49:02,905 [s1-io-3] DEBUG io.netty.buffer.PoolThreadCache  - Freed 14 thread-local buffer(s) from thread: s1-io-3
16:49:02,906 [s1-io-2] DEBUG io.netty.buffer.PoolThreadCache  - Freed 15 thread-local buffer(s) from thread: s1-io-2
16:49:03,056 [main] INFO  com.datastax.oss.driver.internal.core.cql.CqlPrepareAsyncProcessor  - Adding handler to invalidate cached prepared statements on type changes
16:49:03,057 [s2-admin-0] INFO  com.datastax.oss.driver.internal.core.time.Clock  - Using native clock for microsecond precision
16:49:03,291 [main] INFO  com.ing.data.cassandra.jdbc.CassandraConnection  - Connected to cluster: Test Cluster, with session: s2
16:49:03,292 [main] INFO  com.ing.data.cassandra.jdbc.CassandraConnection  - Datacenter: datacenter1; Host: localhost/<unresolved>:9042; Rack: rack1
16:49:03,292 [main] INFO  com.ing.data.cassandra.jdbc.CassandraConnection  - Node: localhost/<unresolved>:9042 runs Cassandra v.4.0.4
16:49:03,305 [main] INFO  giis.tdrules.store.rdb.SchemaReaderJdbc  - Database Platform Info: Database: Cassandra - Version: 4.0.4 - Driver: Cassandra JDBC Driver - Version: 4.12.0
16:49:03,356 [main] ERROR giis.tdrules.store.rdb.SchemaReaderJdbc  - SchemaReaderJdbc.getTableList: 
java.sql.SQLFeatureNotSupportedException: The Cassandra implementation does not support this method.
	at com.ing.data.cassandra.jdbc.CassandraDatabaseMetaData.getImportedKeys(CassandraDatabaseMetaData.java:392)
	at giis.tdrules.store.rdb.SchemaReaderJdbc.readMetadataFks(SchemaReaderJdbc.java:605)
	at giis.tdrules.store.rdb.SchemaReaderJdbc.readBaseTable(SchemaReaderJdbc.java:434)
	at giis.tdrules.store.rdb.SchemaReaderJdbc.readTable(SchemaReaderJdbc.java:323)
	at giis.tdrules.store.rdb.SchemaReader.readTable(SchemaReader.java:197)
	at giis.tdrules.client.rdb.DbSchemaApi.writeTable(DbSchemaApi.java:121)
	at giis.tdrules.client.rdb.DbSchemaApi.getSchema(DbSchemaApi.java:86)
	at giis.tdrules.client.rdb.DbSchemaApi.getSchema(DbSchemaApi.java:73)
	at in2test.application.qagrow.analyze.SchemaLoader.schemaAnalysis(SchemaLoader.java:54)
	at test4in2test.test.UtilTestCassandraGrow.loadSchema(UtilTestCassandraGrow.java:257)
	at test4in2test.test.TestUnitCassandra.testTargetTables(TestUnitCassandra.java:73)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:578)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:93)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:40)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:529)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:756)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:452)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)

Initial information about DB metadata:
INFO giis.tdrules.store.rdb.SchemaReaderJdbc - Database Platform Info: Database: Cassandra - Version: 4.0.4 - Driver: Cassandra JDBC Driver - Version: 4.12.0

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.