categoricaldata / cql Goto Github PK
View Code? Open in Web Editor NEWCategorical Query Language IDE
Home Page: http://categoricaldata.net
Categorical Query Language IDE
Home Page: http://categoricaldata.net
Possible swing bug.
Is there an option to enable logging to stdout when running via the command line?
Is there a way to work with sqlite databases.
Wondered if there is a doc/tutorial to get started? Thanks
I have a large schema in postgres I'm trying to import but seem to be unable to set the timeout option. The entirety of my CQL file is:
options
timeout = 6000
gui_rows_to_display = 600
gui_max_table_size = 6000
gui_max_graph_size = 6000
gui_max_string_size = 6000
always_reload = true
schema sourcedata = import_jdbc_all "jdbc:postgresql://foo/bar"
This seems to always throw:
Timeout after 5 seconds.
Possible solution: add options timeout=X where X > 5 is how many seconds to wait.
If I connect to the baz
db on the same server, I'm able to retrieve the schema, presumably because it is able to query the data in under 5 seconds.
Am I doing something wrong?
EDIT: having just written the above, I stumbled on the static_timeout
option and setting that = 6000
seems to have made progress. However, I now get a Null pointer runtime error:
$ java -Xms2048m -Xmx4096m -cp "postgresql-42.2.14.jar:./CQL.jar" catdata.ide.IDE
objc[4173]: Class FIFinderSyncExtensionHost is implemented in both /System/Library/PrivateFrameworks/FinderKit.framework/Versions/A/FinderKit (0x7fff8d9173d8) and /System/Library/PrivateFrameworks/FileProvider.framework/OverrideBundles/FinderSyncCollaborationFileProviderOverride.bundle/Contents/MacOS/FinderSyncCollaborationFileProviderOverride (0x166e32f50). One of the two will be used. Which one is undefined.
Warning: Nashorn engine is planned to be removed from a future JDK release
java.lang.RuntimeException
at catdata.sql.SqlTable.validate(SqlTable.java:40)
at catdata.sql.SqlSchema.validate(SqlSchema.java:38)
at catdata.sql.SqlSchema.<init>(SqlSchema.java:155)
at catdata.aql.exp.SchExpJdbcAll.eval0(SchExpJdbcAll.java:146)
at catdata.aql.exp.SchExpJdbcAll.eval0(SchExpJdbcAll.java:1)
at catdata.aql.exp.Exp.eval(Exp.java:229)
at catdata.aql.exp.Exp.eval_static(Exp.java:145)
at catdata.aql.exp.AqlStatic.lambda$0(AqlStatic.java:44)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:835)
Anomaly: please report. (Null pointer)
java.lang.RuntimeException: Anomaly
at catdata.aql.gui.AqlCodeEditor.lambda$2(AqlCodeEditor.java:375)
at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:313)
at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:770)
at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:721)
at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:715)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:389)
at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:740)
at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)
java.util.concurrent.ExecutionException: java.lang.RuntimeException
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:205)
at catdata.Util.timeout(Util.java:85)
at catdata.aql.exp.AqlMultiDriver.call(AqlMultiDriver.java:402)
at java.base/java.lang.Thread.run(Thread.java:835)
Caused by: java.lang.RuntimeException
at catdata.sql.SqlTable.validate(SqlTable.java:40)
at catdata.sql.SqlSchema.validate(SqlSchema.java:38)
at catdata.sql.SqlSchema.<init>(SqlSchema.java:155)
at catdata.aql.exp.SchExpJdbcAll.eval0(SchExpJdbcAll.java:146)
at catdata.aql.exp.SchExpJdbcAll.eval0(SchExpJdbcAll.java:1)
at catdata.aql.exp.Exp.eval(Exp.java:229)
at catdata.aql.exp.AqlMultiDriver.lambda$3(AqlMultiDriver.java:402)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
... 1 more
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException
at catdata.Util.timeout(Util.java:93)
at catdata.aql.exp.AqlMultiDriver.call(AqlMultiDriver.java:402)
at java.base/java.lang.Thread.run(Thread.java:835)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:205)
at catdata.Util.timeout(Util.java:85)
... 2 more
Caused by: java.lang.RuntimeException
at catdata.sql.SqlTable.validate(SqlTable.java:40)
at catdata.sql.SqlSchema.validate(SqlSchema.java:38)
at catdata.sql.SqlSchema.<init>(SqlSchema.java:155)
at catdata.aql.exp.SchExpJdbcAll.eval0(SchExpJdbcAll.java:146)
at catdata.aql.exp.SchExpJdbcAll.eval0(SchExpJdbcAll.java:1)
at catdata.aql.exp.Exp.eval(Exp.java:229)
at catdata.aql.exp.AqlMultiDriver.lambda$3(AqlMultiDriver.java:402)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
... 1 more
catdata.LineException: java.util.concurrent.ExecutionException: java.lang.RuntimeException
at catdata.aql.exp.AqlMultiDriver.call(AqlMultiDriver.java:437)
at java.base/java.lang.Thread.run(Thread.java:835)
Setting up a span and attaching morphisms takes just as many lines of code. For this to be smaller, we would need to special case pushouts from colimits by sweeping the diagram under the hood.
--
In the built-in Demo example, the schema colimit could also be made through the inclusion functors that come for free from the Import of amphibian into the two other schemas. Would you want to include another more streamlined colimit that doesn't require writing all the stuff (equations, etc.) by hand?
I had thought that I understood that the syntax of observation_equations
in schema_colimit foo = quotient SchemaA SchemaB
should use underscores to prefix each symbol:
entity_equations
SchemaA.entityA = SchemaB.entityC
observation_equations
forall x. x.SchemaA_entityA_attributeFoo = x.SchemaB_entityC_attributeBar
However, in a currently less trivial colimit
I seem to have to define symbols at arbitrary levels of qualification otherwise I'm given "well-sorted" errors:
schema_colimit Colimit_2014_2019 = quotient SColimit20142017 + SColimit20182019 : tsql {
[...]
observation_equations
forall x. x.toHolderName = x.topHolderName
forall x. x.LAR_activity_year = x.LAR_activity_year
forall x. x.SColimit20142017_LAR_state_code = x.SColimit20182019_LAR_state_code
[...]
Cannot infer a well-sorted term for SColimit20142017_LAR_state_code(x).
Undefined (or not java-parseable) symbol: SColimit20142017_LAR_state_code.
Available symbols:
LAR_state_code
in the above above case, if I omit the schema qualification, CQL seems to be happy. Should I be doing something differently?
Possible RSyntaxArea bug.
The real issue is hyperlinking.
The current CQL randomizer (random instance) generates models; it is designed for large sizes with few or no constraints. It would be useful to have the ability to create random presentations, which must be small instances, but can be highly constrained.
When selecting text, if I get to the bottom of the file, the text drops. Adding an empty line fixes this, but it's weird.
How to reproduce:
Open CQL IDE, open Example "Tutorial", hit run, click on "typeside Ty". The Output will be blank and when you click on either "Decide Equation-in-ctx" or "Normalize Term-in-ctx" you will get the error below:
org.jparsec.error.ParserException: line 1, column 1:
**** Possible problem: only identifiers allowed in . notation (lest left-recusion ensue)
expected, EOF encountered.
The relevant code from "Tutorial" is:
typeside Ty = literal {
java_types
String = "java.lang.String"
Integer = "java.lang.Integer"
java_constants
String = "return input[0]"
Integer = "return java.lang.Integer.parseInt(input[0])"
java_functions
plus : Integer,Integer -> Integer = "return (input[0] + input[1]).intValue()"
}
Given the rest of the Tutorial works correctly I think this might not be a parse error, instead it might be some issue with "Decide Equation-in-ctx" and "Normalize Term-in-ctx".
Regardless this is related to #14.
Sorry if this is not the place for these questions, but I couldn't figure this out from the examples.
I have a dataset from postgres that has a number of columns defined as non-sequential, integer-valued enumerations. Elsewhere, there is documentation on each of element of these enumerations. How can I import the data from jdbc and ensure that the appropriate generators use the correct id
values for the correct multi equations?
For example, say I had the following:
schema S = literal : sql {
entities
Foo
Bar
foreign_keys
bar_type : Foo -> Bar
attributes
bar_type_description : Bar -> Varchar
}
instance J = import_jdbc "jdbc:postgresql://localhost/foo" : S {
Foo ->
"
SELECT row_number() OVER () as id
, cast(bar_type as integer) as bar_type
from foo
"
Bar -> "
SELECT cast(bar_type as integer) as id,
null as bar_type_description
from foo
"
}
instance BarTypes = literal : S {
imports
J
generators
b1 b3 b1111 : Bar
multi_equations
bar_type_description -> {b1 Bur, b3 Bhar, b1111 Bargh}
}
And in the data bar_type
is only ever one of 1, 3, 1111
; I'd like to make sure that the multi equations provide the correct string for the correct id.
The command line interface to CQL does not topologically sort the definitions in the input file before execution, as the GUI executor does.
So you save to the last folder you opened from.
When I run java -jar cql.jar
, I get:
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.UnsupportedClassVersionError: catdata/ide/IDE has been compiled by a more recent version of the Java Runtime (class file version 54.0), this version of the Java Runtime only recognizes class file versions up to 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:495)
javac -version
javac 12.0.1
java -version
openjdk version "1.8.0_212"
OpenJDK Runtime Environment (build 1.8.0_212-b04)
OpenJDK 64-Bit Server VM (build 25.212-b04, mixed mode)
Since the website says "Requires Java 12. Updated June 25, 2019 01:30 pm" I assume it should work with Java 12.
Sorry if this is a naive question, but I'm wondering why the following schema throws an error:
schema S = literal : empty {
entities
A
B
C
D
foreign_keys
x : A -> C
x : B -> C
y : A -> D
y : B -> D
y : C -> D
path_equations
A.x.y = A.y
B.x.y = B.y
}
In equation A.x.y = A.y, Cannot infer a unique well-sorted term for y(x(v)) = y(v).
Candidates: FK v.x[class catdata.aql.exp.Fk].y[class catdata.aql.Term] = FK v[class catdata.aql.exp.Fk].y[class catdata.aql.Term]
FK v.x[class catdata.aql.exp.Fk].y[class catdata.aql.Term] = FK v[class catdata.aql.exp.Fk].y[class catdata.aql.Term]
I'm using the 2019-08-23 version of CQL.
Formal errata - issues that will not be fixed and which users should be aware of - will be listed here:
https://github.com/CategoricalData/CQL/wiki/Errata
After running a file all instance
, schema
and typeside
have a "DP" tab with a button for "Decide Equation-in-ctx" and Input box. Would it be possible to elaborate on what this is and the syntax the input expects in the wiki?
The same goes for the Sat
tab for each instance
. I'm guessing its for setting the type (entity
?) all to one value. Not sure about the syntax here either.
Any information would be appreciated.
It's too difficult to understand what is going on when there are only meaningless identifiers around.
Complete algorithm: http://web.cecs.pdx.edu/~mpj/pubs/polyrec.html
Howdy,
I get 403 (unauthorized) when I try to build this repository... from the vanderbilt releases repository, is it possible to have this build using publicly accessible maven repositories?
Best,
Seve
There should be an additional entry point for the same multi-threaded, out-of-order execution engine that the UI uses; the current command line uses singly-threaded in-order execution.
So e.name = e is not allowed even if e:Employee and name:Employee->String and e:String can be expressed. To work around, write e.name = e@String.
Howdy!
A lot of the examples use single letter variables, e.x.
instance I = literal : S {
generators
a b c : Employee
m s : Department
equations
first(a) = Al
first(b) = Bob last(b) = Bo
first(c) = Carl
name(m) = Math name(s) = CS
age(a) = age(c)
manager(a) = a manager(b) = c manager(c) = c
worksIn(a) = m worksIn(b) = s worksIn(c) = s
secretary(s) = c secretary(m) = a
secretary(worksIn(a)) = manager(a)
worksIn(a) = worksIn(manager(a))
age(a) = "1"
age(b) = "5"
options
prover_simplify_max = 64
}
I have no idea what a
, b
, and c
are in this context. It would be nice for at least me, as a learner, to have longer more descriptive names even if they seem redundant or silly at the expert level.
Another example, I have no idea what Ty
is supposed to be, is it just a Type
or is it some other abstraction?
I'm super happy to help with this I just don't quite know what they should be myself (I'm real new to category theory) or I'd make a PR with some suggestions...
Howdy,
It'd be nice there was a tmBundle (textmate bundle) for hightling cql
syntax. It would allow nearly every text editor on the planet to then highlight CQL
.
Best,
Seve
P.S. I've been looking around to see if there's a tool that'll take the antlr4 code and automagically output one but I haven't yet gotten the parser to build.
Hi!
This project is great! It seems to be very advanced, I will try to play with it. If you are interested in growing this as an open source project, there are a few things that could be improved:
LICENSE
file which contains the stated license (AGPLv3), this will let GitHub (and many other tools) detect the license of this software;.jar
files stored in the repository: Maven/Gradle can already fetch them from the project configuration. When using an IDE, just use plugins to integrate with one of these build systems. That makes Eclipse project creation much easier (the configuration is directly derived from the build system config);These little habits should make development much more pleasant, and they will lower the bar for other contributors (be they from your team or external).
is thrown by
command store1 = export_jdbc_instance CoProd "jdbc:postgresql://localhost/foo" ""
I'm using the following in my typeside:
java_types
Integer = "java.lang.Integer"
Double = "java.lang.Double"
Varchar = "java.lang.String"
java_constants
Integer = "return java.lang.Integer.parseInteger(input[0])"
Double = "return java.lang.Double.parseDouble(input[0])"
Varchar = "return input[0]"
Should I be using a different type or should CQL tell the JDBC posgres driver to use "double precision" (instead of "double")?
Is there a recommended way of handling nulls in an import where I've specified a value should be a foreign key in a schema?
I ran into an exception (that I assume should just be an error) when playing around with the "Query" Example code. The error I get is reproduced below:
Error in Q: java.util.concurrent.ExecutionException: catdata.LocException
The only thing I changed was man
to bla
in query Q
, entity GoodMatch
, after foreign_keys
, line 38 (foreign_keys for_man -> {bla -> m}
). I wanted to know if man
had any special significance since it is not an attribute
nor a foreign_keys
of either schema. The code that causes the error is reproduced below:
(If possible I would like to know what the identifier man
refers to and what foreign_keys for_man -> {man -> m}
means. I assume it is saying the foreign_keys
for_man
maps to the selected m of type Man that satisfies the where clause.)
typeside Ty = literal {
java_types
String = "java.lang.String"
Bool = "java.lang.Boolean"
java_constants
String = "return input[0]"
Bool = "return java.lang.Boolean.parseBoolean(input[0])"
}
schema Source = literal : Ty {
entities
Man Woman
attributes
fav_book_m : Man -> String
fav_book_w : Woman -> String
name_m : Man -> String
name_w : Woman -> String
paying : Man -> Bool
}
schema Target = literal : Ty {
entities
Male GoodMatch PayingGoodMatch
foreign_keys
is_a : PayingGoodMatch -> GoodMatch
for_man : GoodMatch -> Male
attributes
man_name : Male -> String
woman_name : GoodMatch -> String
}
query Q = literal : Source -> Target {
entity
GoodMatch -> {
from m:Man w:Woman
where fav_book_m(m) = fav_book_w(w)
attributes woman_name -> name_w(w)
foreign_keys for_man -> {bla -> m}
}
entity
Male -> {
from man:Man
attributes man_name -> name_m(man)
}
entity
PayingGoodMatch -> {
from man:Man woman:Woman
where fav_book_m(man) = fav_book_w(woman)
paying(man) = true
foreign_keys is_a -> {m -> man w -> woman}
}
}
instance I = literal : Source {
generators
a d e g : Woman
b c f h : Man
multi_equations
paying -> {b true, c false, f true, h true}
name_m -> {b bob, c charlie, f frank, h henry}
name_w -> {a alice, d doris, e ellie, g gina}
fav_book_m -> {b book1, c book1, f book2, h book3}
fav_book_w -> {a book1, d book2, e book2, g book4}
}
instance J = eval Q I
instance I0 = literal : Source {
generators
a0 d0 e0 : Woman
b0 c0 f0 : Man
multi_equations
paying -> {b0 true, c0 false, f0 true}
name_m -> {b0 bob, c0 charlie, f0 frank}
name_w -> {a0 alice, d0 doris, e0 ellie}
fav_book_m -> {b0 book1, c0 book1, f0 book2}
fav_book_w -> {a0 book1, d0 book2, e0 book2}
}
transform h = literal : I0 -> I {
generators
a0 -> a d0 -> d e0 -> e b0 -> b c0 -> c f0 -> f
}
transform k = eval Q h
transform kk = coeval Q k
The reason the viewer doesn’t display the way you’d like is because the elements you want to print nicely are not “definitions in the sense of Patrick”. Here is the type algebra after CQL has simplified it. It contains “definitions” - things that the viewer will simplify - and then the equations, which do not inform the viewer. To obtain this presentation, CQL only applies rewrites of the form
generator -> ...
Usually, this is sufficient because the presentations generated “in bulk” from SQL, CSV, or delta/sigma/pi, will have one labelled null per column in the viewer. Applying rewrites where the LHS is a generator are really fast. In this example, we would need to apply a rewrite (and I’m not even entirely sure name(e) is a definition in Patrick’s sense):
name(e) -> ...
Which although definitely possible is known to slow things down quite a bit. I’ll add a ticket noting this discussion; in the interim, my only suggestion for getting the display you want is the add lots more redundant labelled nulls so that the type algebra simplifier has more rules of the form
generator -> ...
To apply.
functions
e : -> G
m : G, G -> G
name : G -> String
inl r : G
inl f : G
equations
name(inl f) = sue
name(((inl f m inl r) m inl r)) = deb
name(((inl r m inl r) m inl r)) = ryan
name((inl r m inl r)) = jon
name((inl f m inl r)) = dave
(((inl r m inl r) m inl r) m inl r) = e
name(e) = bob
name(inl r) = carl
(inl f m inl f) = e
(inl f m inl r) = (((inl r m inl f) m inl f) m inl f)
name((((inl f m inl r) m inl r) m inl r)) = eric
Definitions:
inr (0, posname) -> name(inl f)
inr (1, posname) -> name((inl f m inl r))
inr (0, pos) -> inl f
inr (1, pos) -> (inl f m inl r)
Original Sks:{inr (1, pos), inr (0, pos), inl r, inr (0, posname), inr (1, posname), inl f}
options
prover = completion
require_consistency = false
typeside Grp = literal {
types
G
String
constants
e:G
bob sue carl eric ryan deb jon dave : String
functions
m: G, G->G
name: G->String
equations
forall g:G. m(g,e)=g
forall g:G. m(e,g)=g
forall g1 g2 g3:G. m(m(g1,g2),g3)=m(g1,m(g2,g3))
}
schema Group = literal : Grp {
entities
elt
attributes
pos: elt->G
posname: elt->String
observation_equations
forall e. name(pos(e))=posname(e)
}
instance DihedralTypes = literal : Group {
generators
f r: G
equations
m(f,f)=e
m(m(m(r,r),r),r)=e
m(f,r)=m(m(m(r,f),f),f)
name(e)=bob name(r)=carl name(m(r,r))=jon name(m(m(r,r),r))=ryan
name(f)=sue name(m(f,r))=dave name(m(m(f,r),r))=deb name(m(m(m(f,r),r),r))=eric
}
instance myDihedral = literal : Group {
imports
DihedralTypes
generators
x y:elt
equations
x.pos=m(f,r)
y.pos=f
}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.