Must also be changed in the SirixDB repo in all test queries as well as in the Python client.

When running the following code:

declare function local:dummy($test) {
    $test
};

declare function local:dummy() {
    local:dummy("test")
};

local:dummy("test")

I get the expected output (test).

However, when changing the last line to

local:dummy()

I get an error: Error: err:XPST0017: Unknown function: local:dummy().

Expected behavior is that brackit should call the second declaration of local:dummy().

Must also be changed in the SirixDB repo in all test queries as well as in the Python client.

Background on DIAMetrics

DIAMetrics is an end-to-end benchmarking and performance framework for query engines developed by Google.

Componenets

Note that there are more details than mentioned here; this is only as an overview, and if we need to add details about more parts, we can do that further down the line

Workload Extractor:

According to the paper, this component extracts a "representative workload" from a live production workload. "DIAMetrics employs a workload extractor and summarizer, which is a feature-based way to ‘mine’ the query logs of a customer and extract a subset of queries that adequately represent the workload of the customer."
For our current purposes, I feel like the best way we can utilize a component like this is to pinpoint a set of heavy workloads that we can keep a list of and then just run those workloads for the time being. To this end, I am working on a PR that will hopefully bring more XQuery files for us to run against from this repository. I will update this issue with a PR number so that we can keep track of everything.

Data and Query Scrambler

This component aims to help protect sensitive data and create variations of the representative sets to prevent sensitive data leakage. The paper lists off a few ways that they achieve this, but for the time being, we can put less emphasis on this part since we will use this internally for the moment.

Workload Runner

According to the paper, this component "allows users to specify various combinations of workloads and systems to be benchmarked. For instance, we may want to run TPC-H on various query engines over various storage formats to see which storage format is the best option for which engine." The runner can either schedule runs of specific engines or spin up and manage (including cleanup and shutdown) entire engine instances for the runs

Monitoring

There are two parts to this:

1. Visualization Framework - which brings up dashboards
2. Alerting Framework - which compares workload performance to historical data and alerts when there iareconcerns

TODO (more to come as we get further along)

• Merge in more XQuery files from xquerl
• Figure out workloads that do not perform well and add them to brackit
• Extract representative workloads somehow

Implement the JsonCollection interface.

The class is StringSerializer.

We have to integrate Sebastian's work on parallelization (probably in some parts with newer Java language possibilities).

https://github.com/sebbae/brackit/tree/parallel

In Simple.java, for example this line is used:

    QueryContext ctx = new QueryContext();

However, when running the file with

javac -cp target/brackit-0.1.11-SNAPSHOT-jar-with-dependencies.jar src/examples/java/org/brackit/examples/Simple.java

We get the following error:

src/examples/java/org/brackit/examples/Simple.java:57: error: QueryContext is abstract; cannot be instantiated
    QueryContext ctx = new QueryContext();
                       ^

We need a pre commit hook or something like that to auto-format code according to our IntelliJ Java Formatter or the Kotlin standard code formatting rules.

Implementation of the DerefDescendantExpr currently is a copy of an old version of the DerefExpr, which drilled down into arrays and lookups the right operand for the objects in this array. Now the semantics for ->>foobar should be that it drills down into objects as well and to find values of objects, which have a foobar field in a predorder traversal.

Brackit currently offers a "one-shot" query through params or STDIN, but we should provide a simple CLI tool, which can execute several queries in a row.

In the near future with a new Java version we'd be able to simplify AST rewritings considerably: https://gavinray97.github.io/blog/what-good-are-record-patterns

When querying the following JSON (using the sirix rest-api):

[{"key": "hey"}, {"key": 0}]

with the following query:

for $i in bit:array-values(.)
where $i=>key instance of xs:integer and $i=>key eq 0
return $i

I get the following error:

Cannot compare 'xs:string' with 'xs:integer'

However, the AndExpr should short-circuit after evaluating key to not be an instance of xs:integer, rather than throwing an error.

It should be noted that the following query:

for $i in bit:array-values(.)
where $i=>key eq 0 and $i=>key instance of xs:integer
return $i

returns:

{'key': 0}

so it appears that the order of the AndExpr is being reversed.

I'll have to investigate the bug, but it certainly has bugs, depending on the JSON input.

We have to add a switch to enable/disable XQuery syntax parsing and to improve the syntax of JSON queries per default. For instance I'd like to get rid of [[ ]] for array indexes and replace it with the common [ ] syntax. Therefore XPath predicates must have another syntax.

Furthermore, we might think about switching the object field deref expression -> to . and to switch the current context item from . to $$ as in the JSONiq specification stated. Furthermore, literals as true and false should then be parsed as the boolean literal types instead of as XPath axis steps. Furthermore it would be handy if object fields doesn't have to be quoted in this case {"foo":"bar"} and {foo:"bar"} should both be possible.

The fn:deep-equal function should handle JSON items. It's defined in the class https://github.com/sirixdb/brackit/blob/master/src/main/java/org/brackit/xquery/function/fn/DeepEqual.java.

For instance:

xquery version "3.0";
declare namespace db="http://sirix.io/xquery/db";
declare function db:map($func, $list) {
    for $item in $list return $func($item)
};
let $fun := function($x) { $x * $x }
return db:map($fun, 1 to 5)

All contributors profiles will be shown in Contributors section and also it will automatically updating

Implement an in-memory implementation similar to the XML SimpleStore.

We should use Java 15 for brackit