philass / futdb Goto Github PK

The SQL statement parser is written with sqlparse. The library is trash with no documentation. The current is implementation is super hacky trying to work with the library. The whole thing needs to be rewritten :(.

A possible library that provides the necessary functionality is select_parser. Below is a stackoverflow link that discusses sql parsing in python, and might be helpful.

https://stackoverflow.com/questions/16909380/sql-parsing-using-pyparsing/16926585#16926585

When running

pylint repl.py

output is

************* Module repl
repl.py:19:4: W0603: Using the global statement (global-statement)
repl.py:30:8: C0415: Import outside toplevel (futhark) (import-outside-toplevel)
repl.py:62:4: W0603: Using the global statement (global-statement)

Need to change structure of REPL to fix these issues, and may not be trival

Currently harkdb doesn't support Group by statements.

Steps

• Get columns in groupby statement
• Make sure select consists of only agg functions and cols in group by
• Come up with Futhark implementation
• Compile to said Futhark Implementation

The SQL Query

select ID, Price from data where Price > 2 and Price < 4

fails because the implementation doesn't handle a column appearing twice in the where clause. Should be a relatively simple fix.

When running
python3 repl.py < test.txt

Where test.txt has contents

.import test_files/ex.csv data
select ID, Price from data
.exit

results in the following compiler warning

harkdb> ['data']
harkdb> {'select': ['ID', 'Price'], 'from': ['data'], 'where': [], 'having': [], 'group by': []}
futhark pyopencl --library ../futhark/db_sel.fut
/usr/local/lib/python3.7/site-packages/pyopencl/init.py:235: CompilerWarning: Non-empty compiler output encountered. Set the environment variable PYOPENCL_COMPILER_OUTPUT=1 to see more.
"to see more.", CompilerWarning)