landawn / abacusutil Goto Github PK
View Code? Open in Web Editor NEWRelease the power in Java programming
License: Apache License 2.0
Release the power in Java programming
License: Apache License 2.0
I have one use-case, where I have dataset and i want to GroupBy on the basis of one column and after grouping by, I want to sort the list of the value for each key.
For example: I have DataSet of Employee with column as ID, NAME, AGE, DEPT_ID?
I want to group by DEP_ID and then sort then by name.
can you please give some example, how can we achieve this with dataset.
Thank you.
I am having trouble executing queries containing WITH clause or CTE. After debugging I ended up at the following line in SQLExecutor that fails to find any parameters to bind. Tried named and positional parameters. Both resulted in the same.
final int parameterCount = namedSQL.getParameterCount();
Error in case of positional param:
No value specified for parameter 1.' and exception = 'No value specified for parameter 1.'
Error in case of named parameters
PSQLException: ERROR: syntax error at or near ":"
Other SQL's starting with "SELECT" seem to work fine.
My SQL looks something like below, an example structure as the original is much more complex
WITH context as ( SELECT ? as days, ? as curcy FROM DUAL ), transactions as ( SELECT t.* from schema.transactions_tbl t, context c WHERE t.curcy_cd = c.curcy_cd and t.dt <= sysdate - c.days ) SELECT * FROM transactions t WHERE t.status = ''SUCCESS'
I noticed that NamedSQL failed to parse and identify those two bind variables. Am I missing something or the code only works when the SQL starts with SELECT clause.
From http://www.landawn.com/:
int min = IntList.random(10).distinct().average().getAsInt();
The variable should be called avg
or average
.
It seems this library contains a lot OpenJDK code (e.g. https://github.com/landawn/AbacusUtil/blob/master/src/com/landawn/abacus/util/stream/IntStream.java). OpenJDK is licensed under "GPLv2 with Classpath Exception". This basically forces this project to be GPL2 (with or without classpath exception). I had the same issue with a library of mine and the guys over at opensource stackexchange said:
By including the modified version of the OpenJDK Base64 implementation, your library has to be effectively distributed under the GPLv2 license, with or without the classpath exception.
If you keep the classpath exception, then it is possible to use your library in non-free projects.
If you drop the classpath exception, then any project using your library has to obey the GPLv2 restrictions, which means that the sources must be made available under a GPL-compatible open-source license when the project is distributed.
This basically means, many of your user may be in violation when using without providing their source. This is probably an important thing to note in your main readme, as "Apache2 licensing" is very misleading here.
Was looking to create a dataset with empty rows in them but with column headers so that I can render the grid with an empty dataset. The following results in an exception.
DataSet ds1 = N.newDataSet(N.asList("id", "type", "amount"), null); // NullPointerException
DataSet ds1 = N.newEmptyDataSet(); //Doesn't have the column headers.
Any other way to do this?
Thanks
Andy
I found that there are very simple ways to serialize DataSet to csv/json/xml etc. But is there anything that can form a DataSet back again once its sent across the wire. I am looking to transport DataSet directly between components to avoid DTO's entirely.
N.newDataSet or any of the N.from don't seem to support this. I am thinking this must be there somewhere and I am not looking at the right place.
In my usage of dataset with some federated data sources (Oracle and Postgres) the column names could appear in capitals or lowercase depending on the source DB and its configuration. This becomes a problem when these datasets are joined with each other using their respective column names. For instance "id" fails to join with "ID".
I believe Dataset columns names should be case insensitive so that mismatch doesn't occur during joins or referencing the columns with names.
Hi, i have been trying to use the library for the past few days but I have been having a hard time due to limited documentation on the API usage. For example my primary purpose is to use Dataset through JdbcUtil and then apply, join/filter/merge etc on the resulting Dataset.
I guess reading from the DB and populating the Dataset was quick and easy. I was even able to join to Datasets but now I have to
I couldnt even figure out how to iterate the Dataset and pull the values from specific columns and populate my pojo list.
Please let me know if there are examples to follow through and understand the usage semantics of the API.
Your product seems interesting and it can be useful i can see from the features. However, I downloaded your library (all - 1.0.3.jar) to be used locally and even via Maven repo. I faced the following errors when running using Mavan or local:
Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class com.landawn.abacus.parser.JSONParserImpl
at com.landawn.abacus.parser.ParserFactory.createJSONParser(ParserFactory.java:194)
at com.landawn.abacus.util.CSVUtil.(CSVUtil.java:69)
at util.ActIDActNameSrcIpTime.TestAbacus.main(TestAbacus.java:24)
The code in TestAbacus.java is below:
`import com.landawn.abacus.DataSet;
import com.landawn.abacus.util.CSVUtil;
import java.io.File;
public class TestAbacus {
public static void main(String args[]) {
File file = new File("C:\\Projects\\Java\\UserProfile\\data\\Sep_Coumn_1000.csv");
DataSet dataset = CSVUtil.loadCSV(file);
}
}
`
I used Java jdk1.8.0_141 running on NetBeans IDE 8.1 (Build 201510222201). I hope i can try out your project it seems powerful.
Regards,
KP
Hi @landawn
I need your urgent help with below issue.
I am using SQLExecutor, my query looks like this:
"SELECT ID FROM EMPLOYEES WHERE DEP_ID IN (replacMe)"
Now suppose, i want to fetch ids of all employees whose dep_id is 1, 2 or 3.
so i replace string replaceMe with the 3 (?, ?, ?) in the query and query becomes like.
"SELECT ID FROM EMPLOYEES WHERE DEP_ID IN (?, ?, ?)"
When i execute this query it throws error:
SQLException|17041|Missing IN or OUT parameter at index:: 1
can you please help me in this.
I would like to be a contributor on this project , found it today and seems very promising and amazing .
Looking on the commit messages to see what happens there is everywhere just release numbers instead of explaining words . This doesn't help future contributors :(
I have a SQL (as follows) which has few bind variables and one of them is an array (IN clause)
SELECT * from MYTABLE m where m.col1 = ? and m.col2 in (?)
Is there an example which can allow me to do this in SQL Executor?
The following SQL is an example where we update the properties of json data with the following UPDATE statement
update my_table mt
set data = data || '{"QUANTITY": ?, "AMOUNT": ?, "AMT_LOCL": ?}'
where mt.data ->> 'TRAN_ID'::text = ? and
mt.data ->> 'ACCT_ID'::text = ? and
mt.process_id = ?
com.landawn.abacus.exception.AbacusException: can't mix '?' and '#{propName}' in the same sql script
To me it appears, "::text" is being confused with a named bind parameter.
Our SQLExecutor batch inserts are taking a very long time. In the example below for a batch size of 5000 and inserting 19764 rows, it took more than 5 min.
dbSQLExecutor.batchInsert(q.getSql(), SQLExecutor.JdbcSettings.create().setBatchSize(5000)
, dataToBeInserted);
Is there any way to optimize this? We are using default JDBCSettings at this point. Not sure if it supports anything that will help in improving the performance.
Hi,
I am looking out for something that can create a DataSet from a json String.
For example:
DataSet ds =
N.newDataSet(
N.asLinkedHashMap("ID", N.asList(1, 2),
"NAME", N.asList("SAHIL", "NITESH"),
"DATA", N.asList("{\"age\": \"27\", \"FATHER_NAME\": \"DHRAM\"}", "{\"age\": \"26\", \"fatherName\": \"PARAM\"}")));
As shown above, my DATA column contains json and now i want to convert this json to a new DataSet.
is there any quick way in API using which i can achieve this.
Actually I am looking for solution urgently, can you please help
I was trying to figure out, how we can pass in parameter value.
Suppose: i have a query:
"SELECT ID FROM EMPLOYEES WHERE DEP_ID IN ( ? ) and NAME LIKE '%?%' "
DEP_IN :- 1,2,3,4,5,6 and NAME value :- somebody
How can we use SQLExcecutor to run this query?
Hi @landawn,
I am getting IndexOutOfBoundsException whenever i perform following actions:
final DataSet dataSet1 = N.newDataSet(N.asLinkedHashMap(
"ACCT_ID", N.asList("1", "2"),
"ACCT_NM", N.asList(11, 12)));
final DataSet dataSet2= N.newEmptyDataSet(N.asList("ACCT_ID_1","ACCT_NM_1"));
DataSet joinedDataSet = dataSet1.innerJoin(dataSet2, N.asMap("ACCT_ID", "ACCT_ID_1"));
2. updating column in case of empty dataSet:
**Example:**
final DataSet idDataSet = N.newEmptyDataSet(N.asList("ID"));
idDataSet.updateColumn("ID", (Try.Function<String, Object, RuntimeException>) Long::valueOf);
I believe this empty check should be part of the API, otherwise as a consumer everybody have to write this check before using above functions.
This is because of a typo in JdbcUtil.
I created a pull request fixing this problem: #15
DynamoDBExecutor is mentioned in http://www.landawn.com/DynamoDBExecutor_view.html but missing in code base.
Where this class can be found?
I was trying to get to the source for RowDataSet to understand the usage better with respect to fulljoin but couldn't find it in github project.
Is this library, only partially open source?
Hi @landawn
I am trying to perform batch insert but somehow my data is not getting committed in batches.
Like i have passed SQLExecutor.JdbcSetting with batch size as 1000, but if conn is null then it set autoCommit as false and commit data for all batches at once.
can you please suggest something, I need to finish this part urgently.
Thank you in advance.
I couldn't see this in the API but wanted to know if there is away to use bind/named parameters in SQLExecutor?
Hi @landawn
Just a random question came in my mind after working on your api from last 3 monhts,
This api is no doubt beautifully written, so few question came to my mind, i though i should ask you.
you worked delicately on this or just to bring the change you developed this beautiful api.
Thank you. have a good day
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.