License: Other
A library which stores, loads and filters time-series-events and associates them catalogues and dynamic catalogues (filter-based).
As of writing this, updates are done directly on backend-instances.
Currently creating an event (and catalogue) takes time, too much when importing hundreds of events.
We use getattr() to check whether the object (Catalogue or Event) is still viable. Otherwise we raise when accessing an attribute.
If the access is OK the base-class's getattr() is called. Except that the default base class might not have a getattr()-method.
According to pythons docs, it is better to use __getattribute__() method for a better access control: https://docs.python.org/3/reference/datamodel.html#object.__getattribute__
If we add a config file, we could make Author field optional in create event/catalog functions. This makes sense since most use cases involves an user creating events or catalogs from his own machine.
Timetable_archive_20231115_1045.tar.gz
Importing this catalog I get:
In [2]: tscat.import_votable("/home/jeandet/Downloads/Timetable_archive_20231115_1045/THEMIS_MPcrossings_2007-2016_V1.xml")
---------------------------------------------------------------------------
OperationalError Traceback (most recent call last)
File ~/.local/lib/python3.10/site-packages/sqlalchemy/engine/base.py:1910, in Connection._execute_context(self, dialect, constructor, statement, parameters, execution_options, *args, **kw)
1909 if not evt_handled:
-> 1910 self.dialect.do_execute(
1911 cursor, statement, parameters, context
1912 )
1914 if self._has_events or self.engine._has_events:
File ~/.local/lib/python3.10/site-packages/sqlalchemy/engine/default.py:736, in DefaultDialect.do_execute(self, cursor, statement, parameters, context)
735 def do_execute(self, cursor, statement, parameters, context=None):
--> 736 cursor.execute(statement, parameters)
OperationalError: too many SQL variables
The above exception was the direct cause of the following exception:
Right now a method called variable_attributes_as_dict() allows us to get the variable attributes as a dict (and thus their key-names), it would be nice to have the same for fix attributed. (Different of each type of entity)
This will allow using tags and products field or attributes for events and catalogues.
Creating events with start_time=stop_time rises the following errror:
167 elif key == 'stop' and hasattr(self, 'start'):
168 if value <= self.start:
--> 169 raise ValueError("stop date has to be after start date")
170 elif key in ['tags', 'products']:
171 if any(type(v) != str for v in value):
ValueError: stop date has to be after start dateCombining regex-match and comparison in All-filter seems not to work when regex-match-value is empty
is in Catalogue(A) or/and in Catalogue(B) (not for SmartCatalogue in the first version)is in no Catalogue (orphan)is in TrashIs it essentially the same as a Catalogue with the difference that is has the filter-field-set in addition to associated events?
_filtered used at import to see whether this event has been added manually to the catalogue or if it is only a result of its filter/predicateCurrent code allows to create multiple catalogues or events having the same UUID.
Tscat is crashing when a tag contains a comma. The same is true for products and other string-list attributes.
Filtering on these columns has to work with a new In-Predicate
To implement versioning of the whole database (to allow a diff-view or quick undo/redo-mechanism) a history (or snapshot) module (tscat.history) could be implemented which allows the user to create arbitrary but complete snapshots of the whole local database. This module should allow listing and navigating the snapshots. Maybe also allowing to show the difference between 2 snapshots.
SciQLop IPython Console Python 3.11.0 (main, Oct 24 2022, 00:00:00)
[GCC 12.2.1 20220819 (Red Hat 12.2.1-2)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.9.0 -- An enhanced Interactive Python. Type '?' for help.
import pandas as pd
df =
pd.read_csv('/home/michotte/Downloads/start_stop_multi_mp_xing.csv')
import tscat
with tscat.Session() as s:
events = [s.create_event(start, stop, "bayane") for start, stop in
zip(df.start, df.stop)]
-----------------------------------------------------------------------
----
TypeError Traceback (most recent call
last)
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/engine/base.py:1800, in
Connection._execute_context(self, dialect, constructor, statement,
parameters, execution_options, *args, **kw)
1798 conn = self._revalidate_connection()
-> 1800 context = constructor(
1801 dialect, self, conn, execution_options, *args, **kw
1802 )
1803 except (exc.PendingRollbackError, exc.ResourceClosedError):
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/engine/default.py:1077, in
DefaultExecutionContext._init_compiled(cls, dialect, connection,
dbapi_connection, execution_options, compiled, parameters,
invoked_statement, extracted_parameters, cache_hit)
1076 for compiled_params in self.compiled_parameters:
-> 1077 param = [
1078 processors[key](compiled_params[key])
1079 if key in processors
1080 else compiled_params[key]
1081 for key in positiontup
1082 ]
1083 parameters.append(dialect.execute_sequence_format(param))
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/engine/default.py:1078, in <listcomp>(.0)
1076 for compiled_params in self.compiled_parameters:
1077 param = [
-> 1078 processors[key](compiled_params[key])
1079 if key in processors
1080 else compiled_params[key]
1081 for key in positiontup
1082 ]
1083 parameters.append(dialect.execute_sequence_format(param))
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/dialects/sqlite/base.py:1004, in
DATETIME.bind_processor.<locals>.process(value)
1003 else:
-> 1004 raise TypeError(
1005 "SQLite DateTime type only accepts Python "
1006 "datetime and date objects as input."
1007 )
TypeError: SQLite DateTime type only accepts Python datetime and date
objects as input.
The above exception was the direct cause of the following exception:
StatementError Traceback (most recent call
last)
Cell In[4], line 1
----> 1 with tscat.Session() as s:
2 events = [s.create_event(start, stop, "bayane") for start,
stop in zip(df.start, df.stop)]
File ~/sciqlop_env/lib64/python3.11/site-packages/tscat/__init__.py:55,
in Session.__exit__(self, exc_type, exc_value, exc_tb)
54 def __exit__(self, exc_type, exc_value, exc_tb):
---> 55 backend().add_and_flush(self.entities)
File ~/sciqlop_env/lib64/python3.11/site-
packages/tscat/orm_sqlalchemy/__init__.py:274, in
Backend.add_and_flush(self, entity_list)
272 def add_and_flush(self, entity_list: List[Union[orm.Event,
orm.Catalogue]]):
273 self.session.add_all(entity_list)
--> 274 self.session.flush()
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/orm/session.py:3444, in Session.flush(self,
objects)
3442 try:
3443 self._flushing = True
-> 3444 self._flush(objects)
3445 finally:
3446 self._flushing = False
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/orm/session.py:3583, in Session._flush(self,
objects)
3580 transaction.commit()
3582 except:
-> 3583 with util.safe_reraise():
3584 transaction.rollback(_capture_exception=True)
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/util/langhelpers.py:70, in
safe_reraise.__exit__(self, type_, value, traceback)
68 self._exc_info = None # remove potential circular
references
69 if not self.warn_only:
---> 70 compat.raise_(
71 exc_value,
72 with_traceback=exc_tb,
73 )
74 else:
75 if not compat.py3k and self._exc_info and
self._exc_info[1]:
76 # emulate Py3K's behavior of telling us when an
exception
77 # occurs in an exception handler.
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/util/compat.py:211, in raise_(***failed resolving
arguments***)
208 exception.__cause__ = replace_context
210 try:
--> 211 raise exception
212 finally:
213 # credit to
214 #
https://cosmicpercolator.com/2016/01/13/exception-leaks-in-python-2-and-3/
215 # as the __traceback__ object creates a cycle
216 del exception, replace_context, from_, with_traceback
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/orm/session.py:3544, in Session._flush(self,
objects)
3542 self._warn_on_events = True
3543 try:
-> 3544 flush_context.execute()
3545 finally:
3546 self._warn_on_events = False
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/orm/unitofwork.py:456, in
UOWTransaction.execute(self)
454 else:
455 for rec in topological.sort(self.dependencies,
postsort_actions):
--> 456 rec.execute(self)
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/orm/unitofwork.py:630, in
SaveUpdateAll.execute(self, uow)
628 @util.preload_module("sqlalchemy.orm.persistence")
629 def execute(self, uow):
--> 630 util.preloaded.orm_persistence.save_obj(
631 self.mapper,
632 uow.states_for_mapper_hierarchy(self.mapper, False,
False),
633 uow,
634 )
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/orm/persistence.py:245, in save_obj(base_mapper,
states, uowtransaction, single)
233 update = _collect_update_commands(
234 uowtransaction, table, states_to_update
235 )
237 _emit_update_statements(
238 base_mapper,
239 uowtransaction,
(...)
242 update,
243 )
--> 245 _emit_insert_statements(
246 base_mapper,
247 uowtransaction,
248 mapper,
249 table,
250 insert,
251 )
253 _finalize_insert_update_commands(
254 base_mapper,
255 uowtransaction,
(...)
271 ),
272 )
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/orm/persistence.py:1238, in
_emit_insert_statements(base_mapper, uowtransaction, mapper, table,
insert, bookkeeping)
1232 result = connection._execute_20(
1233 statement.values(value_params),
1234 params,
1235 execution_options=execution_options,
1236 )
1237 else:
-> 1238 result = connection._execute_20(
1239 statement,
1240 params,
1241 execution_options=execution_options,
1242 )
1244 primary_key = result.inserted_primary_key
1245 if primary_key is None:
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/engine/base.py:1705, in
Connection._execute_20(self, statement, parameters, execution_options)
1701 util.raise_(
1702 exc.ObjectNotExecutableError(statement),
replace_context=err
1703 )
1704 else:
-> 1705 return meth(self, args_10style, kwargs_10style,
execution_options)
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/sql/elements.py:334, in
ClauseElement._execute_on_connection(self, connection, multiparams,
params, execution_options, _force)
330 def _execute_on_connection(
331 self, connection, multiparams, params, execution_options,
_force=False
332 ):
333 if _force or self.supports_execution:
--> 334 return connection._execute_clauseelement(
335 self, multiparams, params, execution_options
336 )
337 else:
338 raise exc.ObjectNotExecutableError(self)
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/engine/base.py:1572, in
Connection._execute_clauseelement(self, elem, multiparams, params,
execution_options)
1560 compiled_cache = execution_options.get(
1561 "compiled_cache", self.engine._compiled_cache
1562 )
1564 compiled_sql, extracted_params, cache_hit =
elem._compile_w_cache(
1565 dialect=dialect,
1566 compiled_cache=compiled_cache,
(...)
1570 linting=self.dialect.compiler_linting |
compiler.WARN_LINTING,
1571 )
-> 1572 ret = self._execute_context(
1573 dialect,
1574 dialect.execution_ctx_cls._init_compiled,
1575 compiled_sql,
1576 distilled_params,
1577 execution_options,
1578 compiled_sql,
1579 distilled_params,
1580 elem,
1581 extracted_params,
1582 cache_hit=cache_hit,
1583 )
1584 if has_events:
1585 self.dispatch.after_execute(
1586 self,
1587 elem,
(...)
1591 ret,
1592 )
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/engine/base.py:1806, in
Connection._execute_context(self, dialect, constructor, statement,
parameters, execution_options, *args, **kw)
1804 raise
1805 except BaseException as e:
-> 1806 self._handle_dbapi_exception(
1807 e, util.text_type(statement), parameters, None, None
1808 )
1810 if (
1811 self._transaction
1812 and not self._transaction.is_active
(...)
1816 )
1817 ):
1818 self._invalid_transaction()
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/engine/base.py:2124, in
Connection._handle_dbapi_exception(self, e, statement, parameters,
cursor, context)
2122 util.raise_(newraise, with_traceback=exc_info[2], from_=e)
2123 elif should_wrap:
-> 2124 util.raise_(
2125 sqlalchemy_exception, with_traceback=exc_info[2],
from_=e
2126 )
2127 else:
2128 util.raise_(exc_info[1], with_traceback=exc_info[2])
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/util/compat.py:211, in raise_(***failed resolving
arguments***)
208 exception.__cause__ = replace_context
210 try:
--> 211 raise exception
212 finally:
213 # credit to
214 #
https://cosmicpercolator.com/2016/01/13/exception-leaks-in-python-2-and-3/
215 # as the __traceback__ object creates a cycle
216 del exception, replace_context, from_, with_traceback
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/engine/base.py:1800, in
Connection._execute_context(self, dialect, constructor, statement,
parameters, execution_options, *args, **kw)
1797 if conn is None:
1798 conn = self._revalidate_connection()
-> 1800 context = constructor(
1801 dialect, self, conn, execution_options, *args, **kw
1802 )
1803 except (exc.PendingRollbackError, exc.ResourceClosedError):
1804 raise
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/engine/default.py:1077, in
DefaultExecutionContext._init_compiled(cls, dialect, connection,
dbapi_connection, execution_options, compiled, parameters,
invoked_statement, extracted_parameters, cache_hit)
1075 if compiled.positional:
1076 for compiled_params in self.compiled_parameters:
-> 1077 param = [
1078 processors[key](compiled_params[key])
1079 if key in processors
1080 else compiled_params[key]
1081 for key in positiontup
1082 ]
1083
parameters.append(dialect.execute_sequence_format(param))
1084 else:
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/engine/default.py:1078, in <listcomp>(.0)
1075 if compiled.positional:
1076 for compiled_params in self.compiled_parameters:
1077 param = [
-> 1078 processors[key](compiled_params[key])
1079 if key in processors
1080 else compiled_params[key]
1081 for key in positiontup
1082 ]
1083
parameters.append(dialect.execute_sequence_format(param))
1084 else:
File ~/sciqlop_env/lib64/python3.11/site-
packages/sqlalchemy/dialects/sqlite/base.py:1004, in
DATETIME.bind_processor.<locals>.process(value)
994 return format_ % {
995 "year": value.year,
996 "month": value.month,
(...)
1001 "microsecond": 0,
1002 }
1003 else:
-> 1004 raise TypeError(
1005 "SQLite DateTime type only accepts Python "
1006 "datetime and date objects as input."
1007 )
StatementError: (builtins.TypeError) SQLite DateTime type only accepts
Python datetime and date objects as input.
[SQL: INSERT INTO events (uuid, start, stop, author, tags, products,
removed, attributes) VALUES (?, ?, ?, ?, ?, ?, ?, ?)]
[parameters: [{'stop': '2008-08-30 19:00:00', 'start': '2008-08-30
18:00:00', 'author': 'bayane', 'products': [], 'tags': [],
'attributes': {}, 'uuid': '60910139-8b03-4401-954f-69e8d1cf9ab4'}]]
We need a method to get a list of all existing tags (catalogues and/or events) from the library.
Importing a non-existing catalogue can fail, if all events are already present. An empty list is passed to add_events() - this is not allowed as per typechecked.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.