Code Monkey home page Code Monkey logo

module-ballerinai-transaction's Introduction

Ballerina Transaction Internal Module

Build GitHub Last Commit Github issues License codecov GraalVM Check

The transaction internal module is a dependency module which required for Ballerina transactions. This internal module is depend on a few other ballerina std-libs such as http, io, config and system.

Building from the Source

Setting Up the Prerequisites

  1. Download and install Java SE Development Kit (JDK) version 17 (from one of the following locations).

  2. Export Github Personal access token with read package permissions as follows,

     export packageUser=<Username>
     export packagePAT=<Personal access token>
    

Building the Source

Execute the commands below to build from the source.

  1. To build the library:

     ./gradlew clean build
    
  2. To run the integration tests:

     ./gradlew clean test
    
  3. To build the module without tests:

     ./gradlew clean build -x test
    
  4. To debug the tests:

     ./gradlew clean build -Pdebug=<port>
    

Contributing to Ballerina

As an open source project, Ballerina welcomes contributions from the community.

For more information, go to the contribution guidelines.

Code of Conduct

All contributors are encouraged to read the Ballerina Code of Conduct.

Useful Links

module-ballerinai-transaction's People

Contributors

aashikam avatar anupama-pathirage avatar azinneera avatar ballerina-bot avatar buddhiwathsala avatar chamil321 avatar daneshk avatar dilhasha avatar dsplayerx avatar gabilang avatar gayaldassanayake avatar gimantha avatar grainier avatar hindujab avatar kalaiyarasiganeshalingam avatar kaneeldias avatar keizer619 avatar lasinicl avatar ldclakmal avatar maheshika avatar manuranga avatar mohanvive avatar niveathika avatar pcnfernando avatar rdhananjaya avatar rpjayasekara avatar sasindudilshara avatar tharmigank avatar thisaruguruge avatar warunalakshitha avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

module-ballerinai-transaction's Issues

Module build has warnings

Description:
$subject

WARNING [ballerinai/transaction/0.0.0::internal.bal:(40:43,40:105)] usage of construct 'beginTransaction((), transactionBlockId, , TWO_PHASE_COMMIT)' is deprecated
WARNING [ballerinai/transaction/0.0.0::internal.bal:(148:5,148:8)] no such documentable parameter 'err'
WARNING [ballerinai/transaction/0.0.0::service_initiator.bal:(69:5,69:5)] concurrent calls will not be made to this method since the method is not an 'isolated' method
WARNING [ballerinai/transaction/0.0.0::service_participant_2pc.bal:(34:5,34:5)] concurrent calls will not be made to this method since the method is not an 'isolated' method
WARNING [ballerinai/transaction/0.0.0::service_participant_2pc.bal:(96:5,96:5)] concurrent calls will not be made to this method since the method is not an 'isolated' method
WARNING [ballerinai/transaction/0.0.0::transaction_block.bal:(54:23,54:85)] usage of construct 'beginTransaction((), transactionBlockId, , TWO_PHASE_COMMIT)' is deprecated
WARNING [ballerinai/transaction/0.0.0::transaction_block.bal:(80:36,80:87)] usage of construct 'abortTransaction(transactionId, transactionBlockId)' is deprecated
WARNING [ballerinai/transaction/0.0.0::transaction_block.bal:(132:38,132:110)] usage of construct 'registerLocalParticipant(transactionBlockId, committedFunc, abortedFunc)' is deprecated
WARNING [ballerinai/transaction/0.0.0::transaction_block.bal:(136:50,137:67)] usage of construct 'beginTransaction(txnContext.transactionId, transactionBlockId, txnContext.registerAtURL, txnContext.coordinationType)' is deprecated
WARNING [ballerinai/transaction/0.0.0::transaction_block.bal:(139:13,139:46)] usage of construct 'notifyLocalParticipantOnFailure()' is deprecated
WARNING [ballerinai/transaction/0.0.0::transaction_block.bal:(145:27,145:65)] usage of construct 'transactionParticipantWrapper(trxFunc)' is deprecated
WARNING [ballerinai/transaction/0.0.0::transaction_block.bal:(147:13,147:46)] usage of construct 'notifyLocalParticipantOnFailure()' is deprecated
WARNING [ballerinai/transaction/0.0.0::transaction_block.bal:(162:38,162:83)] usage of construct 'registerRemoteParticipant(transactionBlockId)' is deprecated
WARNING [ballerinai/transaction/0.0.0::transaction_block.bal:(166:50,167:67)] usage of construct 'beginTransaction(txnContext.transactionId, transactionBlockId, txnContext.registerAtURL, txnContext.coordinationType)' is deprecated
WARNING [ballerinai/transaction/0.0.0::transaction_block.bal:(169:13,169:47)] usage of construct 'notifyRemoteParticipantOnFailure()' is deprecated
WARNING [ballerinai/transaction/0.0.0::transaction_block.bal:(187:23,187:74)] usage of construct 'abortTransaction(transactionId, transactionBlockId)' is deprecated
WARNING [ballerinai/transaction/0.0.0::transaction_block.bal:(188:5,188:53)] usage of construct 'notifyResourceManagerOnAbort(transactionBlockId)' is deprecated

This is also seen in the subsequent module builds too and is noisy.

JDBC Connection is not correctly returned to Connection Pool if the transaction rolls back

Description:

JDBC Connections to an SQLite DB used in a transaction that is rolled back (or connections outside of transactions) are not correctly returned to the connection pool.

Steps to reproduce:

Create a small connection pool and repeatedly perform queries in failing transactions (or outside of transactions).
See the code example for code reproducing this problem.

Minimal code example (click to expand)
import ballerina/io;
import ballerina/sql;
import ballerinax/java.jdbc;

sql:ConnectionPool pool = { 
    maxOpenConnections: 2, // extra small connection pool to see issues faster
    maxConnectionLifeTime: 18,
    minIdleConnections: 1
};

final jdbc:Client testDB = check new jdbc:Client("jdbc:sqlite:test.db", connectionPool = pool);

type Test record {int number;};

public isolated transactional function queryTest(int number) returns int|error {
    var result = testDB->query(`SELECT ${number} AS number;`, Test);
    var count = result.next();
    check result.close();  // closing the stream has no effect on this problem
    if !(count is error) {
        return count.value.number;
    } else {
        return count;
    }
}

public isolated function testFailingTransactions() returns error? {
    foreach var i in 1...5 { // fails after 2 iterations (both connections exhausted)
        transaction {
            io:println("Attempt nr. ", i);
            var result = queryTest(i);
            io:println(result);
            'transaction:setRollbackOnly(error("Abort")); // comment this out to check normal execution
            check commit;
        } on fail error e {
            io:println(e);
        }
    }
}

public function main() {
    var err = testFailingTransactions();
    io:println(err);
    runtime:sleep(60); // wait for connection pool to recover
    err = testFailingTransactions();
    io:println(err);
}

Output:

Running executable

[ballerina/http] started HTTP/WS listener 172.29.205.106:35049
Attempt nr. 1
1
error Error ("Abort",error("Abort"))
Attempt nr. 2
2
error Error ("Abort",error("Abort"))
Attempt nr. 3
error DatabaseError ("Error while executing SQL query: SELECT  ?  AS number;. error while getting the connection for SQLClientConnector. HikariPool-1 - Connection is not available, request timed out after 30001ms..",errorCode=0,sqlState=null)
error Error ("Abort",error("Abort"))
Attempt nr. 4
error DatabaseError ("Error while executing SQL query: SELECT  ?  AS number;. error while getting the connection for SQLClientConnector. HikariPool-1 - Connection is not available, request timed out after 30000ms..",errorCode=0,sqlState=null)
error Error ("Abort",error("Abort"))
Attempt nr. 5
error DatabaseError ("Error while executing SQL query: SELECT  ?  AS number;. error while getting the connection for SQLClientConnector. HikariPool-1 - Connection is not available, request timed out after 30000ms..",errorCode=0,sqlState=null)
error Error ("Abort",error("Abort"))

Attempt nr. 1
1
error Error ("Abort",error("Abort"))
Attempt nr. 2
2
error Error ("Abort",error("Abort"))
Attempt nr. 3
error DatabaseError ("Error while executing SQL query: SELECT  ?  AS number;. error while getting the connection for SQLClientConnector. HikariPool-1 - Connection is not available, request timed out after 30000ms..",errorCode=0,sqlState=null)
error Error ("Abort",error("Abort"))
Attempt nr. 4
error DatabaseError ("Error while executing SQL query: SELECT  ?  AS number;. error while getting the connection for SQLClientConnector. HikariPool-1 - Connection is not available, request timed out after 30000ms..",errorCode=0,sqlState=null)
error Error ("Abort",error("Abort"))
Attempt nr. 5
error DatabaseError ("Error while executing SQL query: SELECT  ?  AS number;. error while getting the connection for SQLClientConnector. HikariPool-1 - Connection is not available, request timed out after 30000ms..",errorCode=0,sqlState=null)
error Error ("Abort",error("Abort"))

Affected Versions:
Swan Lake Beta 2 (slbeta2)

OS, DB, other environment details and versions:

OS: Ubuntu 20.04 running inside WSL2

JDBC Connector used:

[[platform.java11.dependency]]
artifactId="sqlite-jdbc"
version="3.36.0.1"
groupId="org.xerial"

Related Issues (optional):

Suggested Labels (optional):

Suggested Assignees (optional):

Improve build tasks

Description:
$subject Dependencies toml needs to be reverted back to generalise format after a release, which is as of now is not done here. See commit
a5fc7e0

Re-enable `testNewStrandWithTransactionalFunc` test

Description:
testNewStrandWithTransactionalFunc test in transaction_stmt.bal was disabled in 05a43ed. Should fix it and re-enable it.

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Re-enable services test after http is fixed

Description:
Re-enable transactional_services:testTransactionalServices test once the services' issue is resolved in HTTP module

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Distributed test failure

Description:
Distributed tests are failing with the following error

error: {ballerina}TypeCastError {"message":"incompatible types: 'lang.array:ArrayIterator' cannot be cast to 'lang.array:$anonType$_1'"}
	at ballerina.http.1_0_5:getResourceAuthConfig(auth_desugar.bal:118)
	   ballerina.http.1_0_5:getListenerAuthConfig(auth_desugar.bal:95)
	   ballerina.http.1_0_5:authenticateResource(auth_desugar.bal:38)
	   ballerinai.transaction.1_0_4.$anonType$_43:$post$participantMultipleExecution(tests/remote_participant_transaction_test.bal:402)

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Update transaction module with latest task API changes

Description:
Since we have planned to release the task API revamp in the alpha3, need to update the related function to the task module with the latest changes

Latest API to schedule the periodically task :

public isolated function scheduleJobRecurByFrequency(Job job,  decimal interval,  int maxCount = -1, time:Civil? startTime = (), time:Civil? endTime = (), TaskPolicy taskPolicy = {}) returns JobId|Error

HTTP client commits twice

Description:

Steps to reproduce:

Affected Versions:

OS, DB, other environment details and versions:

Related Issues (optional):

Suggested Labels (optional):

Suggested Assignees (optional):

Transaction panics in `endTransaction` with "transaction not found"

Description:
Transactions can panic in endTransaction when many transactions are invoked from multiple threads concurrently.

Steps to reproduce:

Perform many transactions rapidly from multiple threads (e.g. using scheduled tasks)

Minimal code example (click to expand)

The sqlite DB contains a single table "hello" with a string column "world".

import ballerina/io;
import ballerina/sql;
import ballerinax/java.jdbc;
import ballerina/task;
import ballerina/http;

sql:ConnectionPool pool = { 
    maxOpenConnections: 1, 
    maxConnectionLifeTime: 30,
    minIdleConnections: 0
};

final jdbc:Client testDB = check new jdbc:Client("jdbc:sqlite:test.db", connectionPool = pool);

type WorldRecord record {|
    string|() world;
|};

// set simulationError to true to get different stacktrace
public isolated function dbTest(boolean simulateError = false) returns error? {
    transaction {
        stream<WorldRecord, error?> result = testDB->query(`SELECT world FROM hello;`);
        check result.close();
        var _ = check testDB->execute(`INSERT INTO hello VALUES ('test')`);
        result = testDB->query(`SELECT world FROM hello;`);
        check result.close();
        if (simulateError) {
            rollback;
        } else {
            check commit;
        }
    }
    transaction {
        var _ = check testDB->execute(`DELETE FROM hello WHERE world='test'`);
        check commit;
    }
}

public isolated class TestTask {

    *task:Job;
    private int counter = 0;
    private final string name;

    function init(string name) {
        self.name = name;
    }

    public isolated function execute() {
        int count = 0;
        lock {
            count = self.counter.cloneReadOnly();
            self.counter += 1;
        }
        io:println(string`Task: ${self.name},  Invocation:  ${count}`);

        var err = trap dbTest();

        if ! (err is ()) {
            io:println("Error in task " + self.name);
            io:println(err);
            io:println(err.stackTrace());
            panic err; // remove task after first error
        }
    }

    public isolated function schedule(decimal interval) {
        do {
            var _ = check task:scheduleJobRecurByFrequency(self, interval);
        } on fail error err {
            io:println(err);
        }
    }
}


service / on new http:Listener(9097) {
    resource function get .() returns json {
        return {
            hello: "world"
        };
    }
}


public function main() {
    var task1 = new TestTask("a    ");
    var task2 = new TestTask(" b   ");
    var task3 = new TestTask("  c  ");
    var task4 = new TestTask("   d ");
    var task5 = new TestTask("    e");

    decimal interval = 0.1; // adjust to change error occurrence speed
    task1.schedule(interval);
    task2.schedule(interval);
    task3.schedule(interval);
    task4.schedule(interval);
    task5.schedule(interval);
}

Stack Trace:

// error("Transaction: 0afa6c8f-84cd-4b32-91c3-1e0384befbab:3 not found")
[
    {"callableName": "endTransaction", "moduleName": "ballerinai.transaction.0", "fileName": "transaction_block.bal", "lineNumber": 115},
    {"callableName": "dbTest", "moduleName": "test.playground.0", "fileName": "main.bal", "lineNumber": 59},
    {"callableName": "execute", "moduleName": "test.playground.0.TestTask", "fileName": "main.bal", "lineNumber": 101}
]

If simulateError=true the stacktrace is often different:

// error("{ballerina}TypeCastError",message="incompatible types: '()' cannot be cast to '(readonly & lang.transaction:InfoInternal)'")
[
    {"callableName": "info", "moduleName": "ballerina.lang.transaction.0", "fileName": "transaction.bal", "lineNumber": 69},
    {"callableName": "dbTest", "moduleName": "test.playground.0", "fileName": "main.bal", "lineNumber": 76},
    {"callableName": "execute", "moduleName": "test.playground.0.TestTask", "fileName": "main.bal", "lineNumber": 101}
]

The errors seem to suggest that initiatedTransactions is not handled threadsafe everywhere.

Affected Versions:
Ballerina 2201.7.0 (Swan Lake Update 7)

OS, DB, other environment details and versions:
OS: Ubuntu 20.04 running inside WSL2

JDBC Connector used:

[[platform.java11.dependency]]
artifactId="sqlite-jdbc"
version="3.36.0.1"
groupId="org.xerial"

Related Issues (optional):

Suggested Labels (optional):

Suggested Assignees (optional):

Remote participant tests taking too long to execute

Description:
This causes Idle timeout for transactional service tests

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Enable Panic Tests in Transactional Services

Panic test cases have been disabled, because when there is a panic from within a resource function of a service, the service makes a System.exit(). That causes the whole test framework to crash. This needs to be handled, Till then, these test cases have been disabled.

Check feasibility of moving transaction integration tests from distribution repo

Description:
Test in https://github.com/ballerina-platform/ballerina-distribution/blob/master/stdlib-integration-tests/transaction/tests/xa_transactions_test.bal would cover the remote service/participant cases.

Had to ignore remote implementation logic in codecov for the moment.
Refer #299

Steps to reproduce:

Affected Versions:

OS, DB, other environment details and versions:

Related Issues (optional):

Suggested Labels (optional):

Suggested Assignees (optional):

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.