Code Monkey home page Code Monkey logo

stalker's Introduction

Travis-CI Build Status License Supported Python versions PyPI Version Wheel Support

About

Stalker is an Open Source Production Asset Management (ProdAM) Library designed specifically for Animation and VFX Studios but can be used for any kind of projects. Stalker is licensed under LGPL v3.

Features

Stalker has the following features:

  • Designed for Animation and VFX Studios.
  • Platform independent.
  • Default installation handles nearly all the asset and project management needs of an animation and vfx studio.
  • Customizable with configuration scripts.
  • Customizable object model (Stalker Object Model - SOM).
  • Uses TaskJuggler as the project planing and tracking backend.
  • Mainly developed for PostgreSQL in mind but SQLite3 is also supported.
  • Can be connected to all the major 3D animation packages like Maya, Houdini, Nuke, Fusion, Softimage, Blender etc. and any application that has a Python API. And with applications like Adobe Photoshop which does not have a direct Python API but supports win32com or comtypes.
  • Mainly developed for Python 3.0+ and Python 2.7 is fully supported.
  • Developed with TDD practices.

Stalker is build over these other OpenSource projects:

  • Python
  • SQLAlchemy and Alembic
  • Jinja2
  • TaskJuggler

Stalker as a library has no graphical UI, it is a python library that gives you the ability to build your pipeline on top of it. There are other python packages like the Open Source Pyramid Web Application Stalker Pyramid and the Open Source pipeline library Anima which has PyQt/PySide/PySide2 UIs for applications like Maya, Nuke, Houdini, Fusion, Photoshop etc.

Installation

Use:

pip install stalker

Examples

Let's play with Stalker.

Initialize the database and fill with some default data:

from stalker import db
db.setup()
db.init()

Create a User:

from stalker.db.session import DBSession
from stalker import User
me = User(
    name='Erkan Ozgur Yilmaz',
    login='erkanozgur',
    email='[email protected]',
    password='secretpass'
)

# Save the user to database
DBSession.save(me)

Create a Repository for project files to be saved under:

from stalker import Repository
repo = Repository(
    name='Commercial Projects Repository',
    windows_path='Z:/Projects',
    linux_path='/mnt/Z/Projects',
    osx_path='/Volumes/Z/Projects'
)

Create a FilenameTemplate (to be used as file naming convention):

from stalker import FilenameTemplate

task_template = FilenameTemplate(
    name='Standard Task Filename Template',
    target_entity_type='Task',  # This is for files saved for Tasks
    path='{{project.repository.path}}/{{project.code}}/'
         '{%- for parent_task in parent_tasks -%}'
         '{{parent_task.nice_name}}/'
         '{%- endfor -%}',  # This is Jinja2 template code
    filename='{{version.nice_name}}_v{{"%03d"|format(version.version_number)}}'
)

Create a Structure that uses this template:

from stalker import Structure
standard_folder_structure = Structure(
    name='Standard Project Folder Structure',
    templates=[task_template],
    custom_template='{{project.code}}/References'  # If you need extra folders
)

Now create a Project that uses this structure and will be placed under the repository:

from stalker import Project
new_project = Project(
    name='Test Project',
    code='TP',
    structure=standard_folder_structure,
    repositories=[repo],  # if you have more than one repository you can do it
)

Define the project resolution:

from stalker import ImageFormat
hd1080 = ImageFormat(
    name='1080p',
    width=1920,
    height=1080
)

Set the project resolution:

new_project.image_format = hd1080

# Save the project and all the other data it is connected to it
DBSession.save(new_project)

Create Assets, Shots and other Tasks:

from stalker import Task, Asset, Shot, Type

# define Character asset type
char_type = Type(name='Character', code='CHAR', target_entity_type='Asset')

character1 = Asset(
    name='Character 1',
    code='CHAR1',
    type=char_type,
    project=new_project
)

# Save the Asset
DBSession.save(character1)

model = Task(
    name='Model',
    parent=character1
)

rigging = Task(
    name='Rig',
    parent=character1,
    depends=[model],  # For project management, define that Rig can not start
                      # before Model ends.
)

# Save the new tasks
DBSession.save([model, rigging])

# A shot and some tasks for it
shot = Shot(
    name='SH001',
    code='SH001',
    project=new_project
)

# Save the Shot
DBSession.save(shot)

animation = Task(
    name='Animation',
    parent=shot,
)

lighting = Task(
    name='Lighting',
    parent=shot,
    depends=[animation], # Lighting can not start before Animation ends,
    schedule_timing=1,
    schedule_unit='d',  # The task expected to take 1 day to complete
    resources=[me]
)
DBSession.save([animation, lighting])

Let's create versions for the Animation task.

from stalker import Version

new_version = Version(task=animation)
new_version.update_paths()  # to render the naming convention template
new_version.extension = '.ma'  # let's say that we have created under Maya

Let's check how the version path is rendered:

assert new_version.absolute_full_path == \
    "Z:/Projects/TP/SH001/Animation/SH001_Animation_Main_v001.ma"
assert new_version.version_number == 1

Create a new version and check that the version number increased automatically:

new_version2 = Version(task=animation)
new_version2.update_paths()  # to render the naming convention template
new_version2.extension = '.ma'  # let's say that we have created under Maya

assert new_version2.version_number == 2

See more detailed example in API Tutorial.

stalker's People

Contributors

eoyilmaz avatar fredrikaverpil avatar gulsenyilmaz avatar mosari avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

stalker's Issues

Accidentally corrupted db (?) after stalker upgrade

Hi,

I accidentally installed 0.2.14 and performed a db.setup(database_engine_settings) which seems to have corrupted our database which is in sync with 0.2.13.3. It got stuck at "creating the tables" and didn't move on from there:

DEBUG:stalker.db:settings: {'sqlalchemy.pool_size': 1, 'sqlalchemy.echo': False, 'sqlalchemy.url': u'postgresql+psycopg2://dbuser:***@1.2.3.4/stalkerdb', 'sqlalchemy.max_overflow': 3}
DEBUG:stalker.db:engine: Engine(postgresql+psycopg2://dbuser:***@1.2.3.4/stalkerdb)
DEBUG:stalker.db:creating the tables

I'm guessing I should have first performed an alembic upgrade head on the database?

Since time was of the essence (and it's Friday afternoon), I restored a database backup straight away, so all is back on track now with 0.2.13.3. Phew! ๐Ÿ˜…

However, I was wondering... Would it be possible to avoid this kind of thing in the future by perhaps throwing an error of some kind if the tables are not in the expected format, avoiding such issues?

Can't specify "created_by_id" when creating Client object

Hi,

I'd like to register which user created (or updated) a Client in the database:

db.DBSession.add(Client(name=name, created_by_id=created_by_id))

However, it seems that the created_by_id attribute isn't being registered properly (?).
The new Client is created successfully but the SimpleEntities.created_by_id attribute is empty.

Am I doing something wrong ...or is this perhaps not an attribute I should be setting manually like this?
Anyway, I'll use the Client.generic_text attribute for now.

Unicode name causes UnicodeError in Entity

Variable name should be of type str() or unicode(), but I believe that at one point it is converted into a string, which causes an error if it in fact is of type unicode:
name = self._format_name(str(name))

The error I'm getting:

File "//192.168.0.225/Pipeline/bin/python/virtualenvs/python276_amd64_win7_ska
lman/Lib/site-packages\stalker\models\entity.py", line 355, in _validate_name
    name = self._format_name(str(name))
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-2: ordin
al not in range(128)

User.password = <str> vs <unicode>

I've created a UI where the user can enter a new password. With Python 2.7, the new password is read in from the UI as unicode while in Python 3.5, the password is read in as str. It seems that when setting the password in 2.7/unicode, all is fine. But when entering the password in 3.6/str, this is not handled properly and the password is essentially b0rked.

The docs just say "password (str)", but in this case str is two very different things (as they differ in Python 2 vs 3).

Python 2.7 behavior

>>> print(new_password.__class__.__name__)
unicode
>>> some_user.password = new_password

Database entry for the password: aGVsbG8= (correct)

Python 3.5 behavior

>>> print(new_password.__class__.__name__)
str
>>> some_user.password = new_password

Database entry for the password: \x595564... (incorrect)

Cannot import ProjectUser, ClientUser

My code:

from stalker import db, User, Project, Role, DepartmentUser, ProjectUser, ClientUser

Error:

Traceback (most recent call last):
  File "C:\Users\IRUser\Desktop\stalker_manual_fixing.py", line 7, in <module>
    from stalker import db, User, Project, Role, DepartmentUser, ProjectUser, ClientUser
ImportError: cannot import name ProjectUser

...and when removing ProjectUser from the import statement I get the same error with ClientUser:

Traceback (most recent call last):
  File "C:\Users\IRUser\Desktop\stalker_manual_fixing.py", line 7, in <module>
    from stalker import db, User, Project, Role, DepartmentUser, ClientUser
ImportError: cannot import name ClientUser

Dropping support for DB backends other than PostgreSQL

I'm planing to drop DB backend support other than PostgreSQL.

The main reason for that is to greatly optimize the code for only one db backend in mind and stop struggling with keeping the code universal.

There are great data types/instructions/constrains unique to PostgreSQL, but I can't use them just because other backends doesn't support them.

So, before doing that I wanted to here if anyone have any idea about why I shouldn't do that.

Errors when testing test_entity.py

I performed testing of test_entity.py and I received some errors and warnings which I have problems tracking down...
However, I don't think they're related to my latest change (removal of str() conversion for nice_name_in).
I figured I'd just post them here for your knowledge (unless you already are aware of them). I know some of them are a result of me not using psycopg2.

Let me know if I can be of assistance to track these down. Perhaps they are related to my specific setup which is based around Python 2.7 64-bit.

I created a gist with the output of the nosetest here:
https://gist.github.com/fredrikaverpil/3802fa72e792e94cbd20

testing if the default values are updated with the Studio instance ... ok
ERROR
testing the persistence of Asset ... ok
testing if the schedule method will schedule the tasks with the ... ERROR
testing if the schedule method will store the schedule info in ... ERROR
...
testing if tag inits properly ... ok
ERROR
testing if the __auto_name__ class attribute is set to False for ... ok
...
testing if the tasks are correctly scheduled ... ERROR
testing if the tasks are correctly scheduled when the compute ... ERROR
testing if the tasks are correctly scheduled when the compute ... ERROR
testing if the tjp file content is correct ... ok
testing if the tjp file is correctly created ... ok
ERROR
testing if the default value will be used when the dependency_target ... ok

I also receive warnings:

testing if the TaskableEntity part is initialized correctly ... \\192.168.0.225\
Pipeline\source\stalker\stalker\models\task.py:1458: RuntimeWarning: The supplie
d parent and the project is not matching in <Modeling (Task)>, Stalker will use
the parent project (<Test Project 1 (Project)>) as the parent of this Task
  warnings.warn(message, RuntimeWarning)
\\192.168.0.225\Pipeline\source\stalker\st
Warning)
\\192.168.0.225\Pipeline\source\stalker\stalker\models\task.py:1458: RuntimeWarn
ing: The supplied parent and the project is not matching in <Lighting (Task)>, S
talker will use the parent project (<Test Project 1 (Project)>) as the parent of
 this Task
  warnings.warn(message, RuntimeWarning)
ok
testing if the TaskMixin part is initialized correctly ... \\192.168.0.225\Pipel
ine\source\stalker\stalker\models\task.py:1458: RuntimeWarning: The supplied par
ent and the project is not matching in <Modeling (Task)>, Stalker will use the p
arent project (<Test Project1 (Project)>) as the parent of this Task
  warnings.warn(message, RuntimeWarning)
\\192.168.0.225\Pipeline\source\stalker\stalker\models\task.py:1458: RuntimeWarn
ing: The supplied parent and the project is not matching in <Lighting (Task)>, S
talker will use the parent project (<Test Project1 (Project)>) as the parent of
this Task
  warnings.warn(message, RuntimeWarning)
ok
testing if the new task will use the parents project when the given ... \\192.16
8.0.225\Pipeline\source\stalker\stalker\models\task.py:1458: RuntimeWarning: The
 supplied parent and the project is not matching in <New Task (Task)>, Stalker w
ill use the parent project (<Test Project1 (Project)>) as the parent of this Tas
k
  warnings.warn(message, RuntimeWarning)
ok

task.path generates jinja2 error

The following code... (a task which parent is a Project object)...

for task in Task.query.filter_by( id=task_id ):
    print task.path

...generates jinja2.exceptions.UndefinedError: 'version' is undefined error:

Traceback (most recent call last):
  File "\\192.168.0.225\Pipeline\live\skalman\modules\lib\skalmanCommon.py", line 810, in browser_open_task_folder
    print task.path
  File "//192.168.0.225/Pipeline/bin/python/virtualenvs/python276_amd64_win7_skalman/Lib/site-packages\stalker\models\task.py", line 2692, in path
    return jinja2.Template(vers_template.path).render(**kwargs)
  File "//192.168.0.225/Pipeline/bin/python/virtualenvs/python276_amd64_win7_skalman/Lib/site-packages\jinja2\environment.py", line 969, in render
    return self.environment.handle_exception(exc_info, True)
  File "//192.168.0.225/Pipeline/bin/python/virtualenvs/python276_amd64_win7_skalman/Lib/site-packages\jinja2\environment.py", line 742, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "<template>", line 1, in top-level template code
  File "//192.168.0.225/Pipeline/bin/python/virtualenvs/python276_amd64_win7_skalman/Lib/site-packages\jinja2\environment.py", line 397, in getattr
    return getattr(obj, attribute)
jinja2.exceptions.UndefinedError: 'version' is undefined

date_updated attribute off by one hour (we are UTC+1)

Right now, it seems the SimpleEntity.date_updated attribute is using UTC (pytz.utc) and in our case this means all timestamps are off by one hour since we are UTC+1 (Stockholm, Sweden).

If I were to tell Stalker that we are UTC+1, what would be the best approach?

Cannot modify User.companies

I'm getting an error when I attempt to modify User.companies which contains a number of companies:

# Assuming there's already companies registered in stalker
companies = []
for client in Client.query.all():
    companies.append(client)

# Assuming there's already a user in stalker called Peter Griffin, with one or more companies already in the user.companies attribute:
for user in User.query.filter_by(name='Peter Griffin'):
    user.companies = companies

db.DBSession.commit() # <----- triggers the error

The error:

Traceback (most recent call last):
  File "C:\Users\IRUser\Desktop\stalker_manual_fixing.py", line 28, in <module>
    db.DBSession.commit()
  File "//192.168.0.225/Pipeline/bin/python/virtualenvs\python276_amd64_win7_skalman/Lib/site-packages\sqlalchemy\orm\scoping.py", line 150, in do
    return getattr(self.registry(), name)(*args, **kwargs)
  File "//192.168.0.225/Pipeline/bin/python/virtualenvs\python276_amd64_win7_skalman/Lib/site-packages\sqlalchemy\orm\session.py", line 776, in commit
    self.transaction.commit()
  File "//192.168.0.225/Pipeline/bin/python/virtualenvs\python276_amd64_win7_skalman/Lib/site-packages\sqlalchemy\orm\session.py", line 377, in commit
    self._prepare_impl()
  File "//192.168.0.225/Pipeline/bin/python/virtualenvs\python276_amd64_win7_skalman/Lib/site-packages\sqlalchemy\orm\session.py", line 357, in _prepare_impl
    self.session.flush()
  File "//192.168.0.225/Pipeline/bin/python/virtualenvs\python276_amd64_win7_skalman/Lib/site-packages\sqlalchemy\orm\session.py", line 1919, in flush
    self._flush(objects)
  File "//192.168.0.225/Pipeline/bin/python/virtualenvs\python276_amd64_win7_skalman/Lib/site-packages\sqlalchemy\orm\session.py", line 2037, in _flush
    transaction.rollback(_capture_exception=True)
  File "//192.168.0.225/Pipeline/bin/python/virtualenvs\python276_amd64_win7_skalman/Lib/site-packages\sqlalchemy\util\langhelpers.py", line 60, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "//192.168.0.225/Pipeline/bin/python/virtualenvs\python276_amd64_win7_skalman/Lib/site-packages\sqlalchemy\orm\session.py", line 2001, in _flush
    flush_context.execute()
  File "//192.168.0.225/Pipeline/bin/python/virtualenvs\python276_amd64_win7_skalman/Lib/site-packages\sqlalchemy\orm\unitofwork.py", line 372, in execute
    rec.execute(self)
  File "//192.168.0.225/Pipeline/bin/python/virtualenvs\python276_amd64_win7_skalman/Lib/site-packages\sqlalchemy\orm\unitofwork.py", line 481, in execute
    self.dependency_processor.process_saves(uow, states)
  File "//192.168.0.225/Pipeline/bin/python/virtualenvs\python276_amd64_win7_skalman/Lib/site-packages\sqlalchemy\orm\dependency.py", line 558, in process_saves
    uowcommit, False)
  File "//192.168.0.225/Pipeline/bin/python/virtualenvs\python276_amd64_win7_skalman/Lib/site-packages\sqlalchemy\orm\dependency.py", line 575, in _synchronize
    sync.clear(dest, self.mapper, self.prop.synchronize_pairs)
  File "//192.168.0.225/Pipeline/bin/python/virtualenvs\python276_amd64_win7_skalman/Lib/site-packages\sqlalchemy\orm\sync.py", line 57, in clear
    (r, orm_util.state_str(dest))
AssertionError: Dependency rule tried to blank-out primary key column 'Client_Users.uid' on instance '<ClientUser at 0x564d320>'

Defining primary client of a project

In Stalker 0.2.15.1:

New: Project to Client relation is now a many-to-many relation, thus it is possible to set multiple Clients for each project with each client having their own roles in a specific project.

In our production management GUI, we show projects prefixed with the client company name. I'd like to keep this and I'm thinking I can just add a Client.id to Project.generic_text to indicate which Client is the "primary" one, now when there's not a single client associated with a project any longer.

So, I was just wondering if you already had something like this in mind (docs doesn't seem updated) and if there was already a better attribute in place already for this use case?

Python 3 support?

I'm currently developing in Python 3 and will be using stalker. I've been performing a few queries and so far it's working great.

Are you aware of any issues I might stumble upon or should stalker be "Python 3 safe"?

project creation from help file not working anymore

Using the latest version of stalker, this command not working anymore:

new_project = Project(
name='TestProject1',
code='TP1',
structure=standard_folder_structure,
repositories=[repo] # if you have more than one repository you can do it
)

from this page:
https://pypi.org/project/stalker/
error:

TypeError: Project instances can not be initialized without a stalker.models.status.StatusList instance, please pass a suitable StatusList (StatusList.target_entity_type=Project) with the 'status_list' argument

task.task_depends_to always returns an empty list

for task in Task.query.filter_by( id=task_id ):
  print task.task_depends_to

...just prints []. The task in question has a parent task and a child task. The parent task is attached to a Project.

I'm trying to get a list of tasks which depend on the task given.

How to disable DEBUG logging?

How do I disable the DEBUG logging, as this reveals the USER and PASSWORD?

DEBUG:stalker.db:settings: {'sqlalchemy.pool_size': 1, 'sqlalchemy.echo': False, 'sqlalchemy.url': u'postgresql+pg8000://USER:[email protected]/stalker', 'sqlalchemy.max_overflow': 3}
DEBUG:stalker.db:engine: Engine(postgresql+pg8000://USER:***@1.2.3.4/stalker)
DEBUG:stalker.db:creating the tables

Help with SQL query

Hi @eoyilmaz ,

I have the following query:

versions_id_list = [ ... ]
versions = DBSession.query(Version.id, Version.created_with, Version.name,
                            Version.take_name, Version.version_number,
                            Version.created_by_id, Version.updated_by_id,
                            Version.generic_text).filter(
                                Version.id.in_(versions_id_list)).all()

But I would also like to the tags for each version tuple returned (or more preferably, a list of tag IDs). I can't just append Version.tags in that query, as it's not a simple Version attribute.

How would you do recommend I do this without affecting performance too negatively?
I can easily just perform a query for each returned version, but the way I do this currently, it affects performance negatively when dealing with a large set of versions.

Database created with Python 3.5 cannot be read with Python 2.7

Continuing from this issue: #43

When first using stalker I followed the tutorial and used Python 3.5. Then later I tried to connect with Python 2.7 but that fails. It seems like it's related to sqlalchemy which uses pickle to dump and load data. In Python 3.5 the default pickle protocol is 4, while on Python 2.7 the highest supported pickle protocol is 2. It seems at least some data is stored with pickle protocol 4.
I didn't have time yet to investigate thoroughly yet.

For complete reference: running in Python 2.7.13 I do the following:

from stalker import db
db.setup()

And this is the error and traceback:

DEBUG:stalker.db:no settings given, using the default: {'sqlalchemy.echo': False, 'sqlalchemy.url': 'postgresql://localhost:5432'}
DEBUG:stalker.db:settings: {'sqlalchemy.echo': False, 'sqlalchemy.url': 'postgresql://localhost:5432'}
DEBUG:stalker.db:engine: Engine(postgresql://localhost:5432)
DEBUG:stalker.db:current_alembic_version: 0063f547dc2e
DEBUG:stalker.db:creating the tables
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-5-c49e9dc02e34> in <module>()
----> 1 db.setup()

/Users/jasperge/dev/stalker/venv-2.7/lib/python2.7/site-packages/stalker-0.2.17.6-py2.7.egg/stalker/db/__init__.pyc in setup(settings)
     76
     77     # update defaults
---> 78     update_defaults_with_studio()
     79
     80     # create repo env variables

/Users/jasperge/dev/stalker/venv-2.7/lib/python2.7/site-packages/stalker-0.2.17.6-py2.7.egg/stalker/db/__init__.pyc in update_defaults_with_studio()
     89         with DBSession.no_autoflush:
     90             from stalker.models.studio import Studio
---> 91             studio = Studio.query.first()
     92             if studio:
     93                 logger.debug('found a studio, updating defaults')

/Users/jasperge/dev/stalker/venv-2.7/lib/python2.7/site-packages/sqlalchemy/orm/query.pyc in first(self)
   2688             ret = list(self)[0:1]
   2689         else:
-> 2690             ret = list(self[0:1])
   2691         if len(ret) > 0:
   2692             return ret[0]

/Users/jasperge/dev/stalker/venv-2.7/lib/python2.7/site-packages/sqlalchemy/orm/query.pyc in __getitem__(self, item)
   2480                 return list(res)[None:None:item.step]
   2481             else:
-> 2482                 return list(res)
   2483         else:
   2484             if item == -1:

/Users/jasperge/dev/stalker/venv-2.7/lib/python2.7/site-packages/sqlalchemy/orm/loading.pyc in instances(query, cursor, context)
     88     except Exception as err:
     89         cursor.close()
---> 90         util.raise_from_cause(err)
     91
     92

/Users/jasperge/dev/stalker/venv-2.7/lib/python2.7/site-packages/sqlalchemy/util/compat.pyc in raise_from_cause(exception, exc_info)
    201     exc_type, exc_value, exc_tb = exc_info
    202     cause = exc_value if exc_value is not exception else None
--> 203     reraise(type(exception), exception, tb=exc_tb, cause=cause)
    204
    205 if py3k:

/Users/jasperge/dev/stalker/venv-2.7/lib/python2.7/site-packages/sqlalchemy/orm/loading.pyc in instances(query, cursor, context)
     73             if single_entity:
     74                 proc = process[0]
---> 75                 rows = [proc(row) for row in fetch]
     76             else:
     77                 rows = [keyed_tuple([proc(row) for proc in process])

/Users/jasperge/dev/stalker/venv-2.7/lib/python2.7/site-packages/sqlalchemy/orm/loading.pyc in polymorphic_instance(row)
    629             if _instance:
    630                 return _instance(row)
--> 631         return instance_fn(row)
    632     return polymorphic_instance
    633

/Users/jasperge/dev/stalker/venv-2.7/lib/python2.7/site-packages/sqlalchemy/orm/loading.pyc in _instance(row)
    435             _populate_full(
    436                 context, row, state, dict_, isnew, load_path,
--> 437                 loaded_instance, populate_existing, populators)
    438
    439             if isnew:

/Users/jasperge/dev/stalker/venv-2.7/lib/python2.7/site-packages/sqlalchemy/orm/loading.pyc in _populate_full(context, row, state, dict_, isnew, load_path, loaded_instance, populate_existing, populators)
    496
    497         for key, getter in populators["quick"]:
--> 498             dict_[key] = getter(row)
    499         if populate_existing:
    500             for key, set_callable in populators["expire"]:

/Users/jasperge/dev/stalker/venv-2.7/lib/python2.7/site-packages/sqlalchemy/sql/sqltypes.pyc in process(value)
   1476                 if value is None:
   1477                     return None
-> 1478                 return loads(value)
   1479         else:
   1480             def process(value):

ValueError: unsupported pickle protocol: 4

Where to store information about environment?

Hi,

I've previously stored environment variables (which is used for software version control) in the generic_text attribute of the Version SOM.

I now intend to change this approach as multiple Versions store the same environment and I actually think this might make up 50% of all data in our Stalker db by now.

So instead I'd like to be able to store each unique environment into something. Then I'd just save down a reference to this environment on the Version.

Which SOM would you suggest I use to store these environments?
It's plain text and I'm going to process it into a Python dictionary, JSON and YAML.

I first thought maybe I could use the Note SOM and set its type attribute using the Type SOM (for example of type "Environment"). I would then attach this Note to each Version object. Do you think this is a good idea or do you have any other suggestions?

module group_finder missing

when doing a clean install os stalker, then pyramid-stalker, the lastest is saying the group_finder is missing. How to solve this ?

(btw google code is shuting down in a few days, you should export the project to github)

pip install stalker - does not find required modules (sqlalchemy and jinja2)

pip install stalker does not automatically download and install required modules.

(python2.7.9_win7-64_stalker0.2.13) Y:\bin\python\virtualenvs\python2.7.9_win7-6
4_stalker0.2.13\Scripts>pip install -U stalker
Collecting stalker
  Downloading stalker-0.2.13.tar.gz (519kB)
    100% |################################| 520kB 671kB/s
    Traceback (most recent call last):
      File "<string>", line 20, in <module>
      File "c:\users\iruser\appdata\local\temp\pip-build-k2fqn4\stalker\setup.py
", line 5, in <module>
        import stalker
      File "stalker\__init__.py", line 41, in <module>
        from stalker.models.auth import Group, Permission, User, LocalSession, R
ole
      File "stalker\models\auth.py", line 27, in <module>
        from sqlalchemy import (Table, Column, Integer, ForeignKey, String, Date
Time,
    ImportError: No module named sqlalchemy
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

      File "<string>", line 20, in <module>

      File "c:\users\iruser\appdata\local\temp\pip-build-k2fqn4\stalker\setup.py
", line 5, in <module>

        import stalker

      File "stalker\__init__.py", line 41, in <module>

        from stalker.models.auth import Group, Permission, User, LocalSession, R
ole

      File "stalker\models\auth.py", line 27, in <module>

        from sqlalchemy import (Table, Column, Integer, ForeignKey, String, Date
Time,

    ImportError: No module named sqlalchemy

    ----------------------------------------
    Command "python setup.py egg_info" failed with error code 1 in c:\users\irus
er\appdata\local\temp\pip-build-k2fqn4\stalker

Query: latest Version object(s) from project

Hi,

I need to create list of all of the Version objects in a certain project - but I'm only interested in the ones with the highest version number.

Today I do it like this:

latest = []
tasks = Task.query.filter_by(project_id=1)
for task in tasks:
    for version in task.versions:
        latest.append(version.latest_version)

However, iterating over the versions in Python has proven to actually be a bit of a performance hit and I believe I would benefit greatly if I were able to instead perform the following kind of query (without having to iterate over task.versions):

latest = Version.query.join(Task).filter(Task.project_id==1).filter(Version.max_version_number==Version.version_number).all()

As far as I can tell, this kind of query is not possible, correct?

I get this error when attempting to run it:

Traceback (most recent call last):
  File "\\192.168.0.226\pipeline\dev\skalman\skalman\uilib\browserfunctions.py", line 41, in <lambda>
    lambda: get_files_data(skalWin, proj_box))
  File "\\192.168.0.226\pipeline\dev\skalman\skalman\uilib\browserfunctions.py", line 354, in get_files_data
    progressbar=skalWin.progressbar)
  File "\\192.168.0.226\pipeline\dev\skalman\skalman\lib\stalkerscripts\stalkerquery.py", line 181, in versions_in_project
    ).filter(Version.max_version_number==Version.version_number
  File "c:\python27\lib\site-packages\sqlalchemy\orm\query.py", line 2588, in all
    return list(self)
  File "c:\python27\lib\site-packages\sqlalchemy\orm\query.py", line 2736, in __iter__
    return self._execute_and_instances(context)
  File "c:\python27\lib\site-packages\sqlalchemy\orm\query.py", line 2751, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "c:\python27\lib\site-packages\sqlalchemy\engine\base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "c:\python27\lib\site-packages\sqlalchemy\sql\elements.py", line 323, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "c:\python27\lib\site-packages\sqlalchemy\engine\base.py", line 1010, in _execute_clauseelement
    compiled_sql, distilled_params
  File "c:\python27\lib\site-packages\sqlalchemy\engine\base.py", line 1146, in _execute_context
    context)
  File "c:\python27\lib\site-packages\sqlalchemy\engine\base.py", line 1341, in _handle_dbapi_exception
    exc_info
  File "c:\python27\lib\site-packages\sqlalchemy\util\compat.py", line 200, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "c:\python27\lib\site-packages\sqlalchemy\engine\base.py", line 1139, in _execute_context
    context)
  File "c:\python27\lib\site-packages\sqlalchemy\engine\default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) can't adapt type 'property' [SQL: 'SELECT "Versions".id AS "Versions_id", "Links".id AS "Links_id", "Entities".id AS "Entities_id", "SimpleEntities".stalker_version AS "SimpleEntities_stalker_version", "SimpleEntities".id AS "SimpleEntities_id", "SimpleEntities".entity_type AS "SimpleEntities_entity_type", "SimpleEntities".name AS "SimpleEntities_name", "SimpleEntities".description AS "SimpleEntities_description", "SimpleEntities".created_by_id AS "SimpleEntities_created_by_id", "SimpleEntities".updated_by_id AS "SimpleEntities_updated_by_id", "SimpleEntities".date_created AS "SimpleEntities_date_created", "SimpleEntities".date_updated AS "SimpleEntities_date_updated", "SimpleEntities".type_id AS "SimpleEntities_type_id", "SimpleEntities".generic_text AS "SimpleEntities_generic_text", "SimpleEntities".thumbnail_id AS "SimpleEntities_thumbnail_id", "SimpleEntities".html_style AS "SimpleEntities_html_style", "SimpleEntities".html_class AS "SimpleEntities_html_class", "Links".original_filename AS "Links_original_filename", "Links".full_path AS "Links_full_path", "Versions".task_id AS "Versions_task_id", "Versions".take_name AS "Versions_take_name", "Versions".version_number AS "Versions_version_number", "Versions".is_published AS "Versions_is_published", "Versions".created_with AS "Versions_created_with", "Versions".parent_id AS "Versions_parent_id" \nFROM "SimpleEntities" JOIN "Entities" ON "SimpleEntities".id = "Entities".id JOIN "Links" ON "Entities".id = "Links".id JOIN "Versions" ON "Links".id = "Versions".id JOIN ("SimpleEntities" AS "SimpleEntities_1" JOIN "Entities" AS "Entities_1" ON "SimpleEntities_1".id = "Entities_1".id JOIN "Tasks" AS "Tasks_1" ON "Entities_1".id = "Tasks_1".id) ON "Tasks_1".id = "Versions".task_id \nWHERE "Tasks_1".project_id = %(project_id_1)s AND "Versions".version_number = %(version_number_1)s'] [parameters: {'version_number_1': <property object at 0x00000000064186D8>, 'project_id_1': 1774}]

SOM and attributes missing in documentation

I noticed that the Client SOM is missing from the pythonhosted documentation.

Also, the User.company, Project.client and SimpleEntity.generic_text attributes are missing from the documentation.

I'm not sure how these should be added and submitted to github. Is there somewhere I can read up on that?

DetachedInstanceError

Is it possible to configure stalker in such a way that it will attempt to reconnect to the db if the connection was for some reason lost?

 File "C:\Users\iruser\skalman\condaenvs\skalman_py27\Lib\site-packages\stalker\models\task.py", line 1200, in __eq__
    and self.project == other.project and self.parent == other.parent \
  File "C:\Users\iruser\skalman\condaenvs\skalman_py27\Lib\site-packages\sqlalchemy\orm\attributes.py", line 237, in __get__
    return self.impl.get(instance_state(instance), dict_)
  File "C:\Users\iruser\skalman\condaenvs\skalman_py27\Lib\site-packages\sqlalchemy\orm\attributes.py", line 584, in get
    value = self.callable_(state, passive)
  File "C:\Users\iruser\skalman\condaenvs\skalman_py27\Lib\site-packages\sqlalchemy\orm\strategies.py", line 530, in _load_for_state
    (orm_util.state_str(state), self.key)
DetachedInstanceError: Parent instance <Task at 0x24aefc39d68> is not bound to a Session; lazy load operation of attribute 'parent' cannot proceed

Hi i have problem when using stalker in maya

1st big Thanks for sharing all this awesome stuff with the vex community!!! Thank you.

hi do you provide a documentation for anima setup ?
because i have problem when using anima and stalker in maya python!!

here is the error

import stalker
Error: line 1: ImportError: file line 1: No module named stalker
import anima
Error: line 1: ImportError: file line 1: No module named anima

http://vfx.animaistanbul.com/stalker_home/
can you share how to setup anima, stalker and stalker_pyramid for a small scale studio pipeline?
im so confuse when using anima!! not sure where to start!! please help

Initial db.init() fails

If I delete the stalker database, create a new one and run db.init() I always get an error the first time. If I simply run db.init() a second time, all tables are created just fine.

This is what happens when I run db.init() the very first time:

DEBUG:stalker.db:no settings given, using the default: {'sqlalchemy.pool_size': 1, 'sqlalchemy.echo': False, 'sqlalchemy.url': 'mysql://user:[email protected]/stalker', 'sqlalchemy.max_overflow': 3}
DEBUG:stalker.db:settings: {'sqlalchemy.pool_size': 1, 'sqlalchemy.echo': False, 'sqlalchemy.url': 'mysql://IRUser:[email protected]/stalker', 'sqlalchemy.max_overflow': 3}
DEBUG:stalker.db:engine: Engine(mysql://IRUser:***@192.168.0.131/stalker)
DEBUG:stalker.db:creating the tables
Traceback (most recent call last):
  File "C:\Users\IRUser\Desktop\python workbench\stalker\working_and_useful\stalker_initdb.py", line 5, in <module>
    db.setup()
  File "\\192.168.0.225\pipeline\source\stalker\stalker\db\__init__.py", line 71, in setup
    Base.metadata.create_all(engine)
  File "C:\Python27\lib\site-packages\sqlalchemy\sql\schema.py", line 3291, in create_all
    tables=tables)
  File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 1547, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 1194, in _run_visitor
    **kwargs).traverse_single(element)
  File "C:\Python27\lib\site-packages\sqlalchemy\sql\visitors.py", line 119, in traverse_single
    return meth(obj, **kw)
  File "C:\Python27\lib\site-packages\sqlalchemy\sql\ddl.py", line 714, in visit_metadata
    _ddl_runner=self)
  File "C:\Python27\lib\site-packages\sqlalchemy\event\attr.py", line 257, in __call__
    fn(*args, **kw)
  File "C:\Python27\lib\site-packages\sqlalchemy\sql\ddl.py", line 243, in __call__
    return bind.execute(self.against(target))
  File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 720, in execute
    return meth(self, multiparams, params)
  File "C:\Python27\lib\site-packages\sqlalchemy\sql\ddl.py", line 67, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 774, in _execute_ddl
    compiled
  File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 947, in _execute_context
    context)
  File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 1108, in _handle_dbapi_exception
    exc_info
  File "C:\Python27\lib\site-packages\sqlalchemy\util\compat.py", line 185, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 940, in _execute_context
    context)
  File "C:\Python27\lib\site-packages\sqlalchemy\engine\default.py", line 435, in do_execute
    cursor.execute(statement, parameters)
  File "C:\Python27\lib\site-packages\MySQLdb\cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "C:\Python27\lib\site-packages\MySQLdb\connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
sqlalchemy.exc.OperationalError: (OperationalError) (1022, "Can't write; duplicate key in table '#sql-dc4_2d5'") 'ALTER TABLE `Pages` ADD CONSTRAINT project_x_id FOREIGN KEY(project_id) REFERENCES `Projects` (id)' ()
[Finished in 5.2s with exit code 1]

pip install stalker - did not work if sqlalchemy and jinja2 wasn't already installed

I just re-installed stalker for Python 2.7.8 in a virtualenv in Windows ...and I noticed that I was unable to install stalker using pip install stalker unless sqlalchemy and jinja2 was installed manually prior to installing stalker.

I know that in the past these packages were downloaded and installed automatically when you installed stalker. Just figured I'd let you know about it.

stalker_pyramid 0.1.0.b1 install error

Hi,
if im doing this with latest python 3.6:
pip install stalker_pyramid
I am having this error:

Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-install-u1ko3rm4/stalker-pyramid/setup.py", line 4, in
import stalker_pyramid
File "/tmp/pip-install-u1ko3rm4/stalker-pyramid/stalker_pyramid/init.py", line 30, in
from stalker.config import defaults
ImportError: cannot import name 'defaults

Any ideas?

Thanks a lot.

Custom defined function on database connection/communication error

I was thinking what should happen in our office if the stalker database for some reason goes offline. This would be a severe error and would require immediate attention.

It would be very good if stalker would trigger a user-defined function whenever it cannot communicate with the database. In my particular case, I would want perform a series of notifcations measures (sms, email). I have also been toying with the of flicking a switch which would attempt to connect to a mirrored version of the stalker database and permanently start using this instead. If this also fails, the production tool which is using stalker would get closed down.

Thoughts?

Issues with alembic upgrade

I'm attempting to upgrade my database which has alembic version 39d3c16ff005. I believe that should correspond to stalker version 0.2.13.

$ alembic upgrade head

INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade 39d3c16ff005 -> 258985128aff, create EntityGroups table
INFO  [alembic.runtime.migration] Running upgrade 258985128aff -> 745b210e6907, fix non existing thumbnails
INFO  [alembic.runtime.migration] Running upgrade 745b210e6907 -> f2005d1fbadc, added ProjectClients
Traceback (most recent call last):
  File "/usr/local/bin/alembic", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/site-packages/alembic/config.py", line 479, in main
    CommandLine(prog=prog).main(argv=argv)
  File "/usr/local/lib/python2.7/site-packages/alembic/config.py", line 473, in main
    self.run_cmd(cfg, options)
  File "/usr/local/lib/python2.7/site-packages/alembic/config.py", line 456, in run_cmd
    **dict((k, getattr(options, k)) for k in kwarg)
  File "/usr/local/lib/python2.7/site-packages/alembic/command.py", line 174, in upgrade
    script.run_env()
  File "/usr/local/lib/python2.7/site-packages/alembic/script/base.py", line 397, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/usr/local/lib/python2.7/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/local/lib/python2.7/site-packages/alembic/util/compat.py", line 79, in load_module_py
    mod = imp.load_source(module_id, path, fp)
  File "alembic/env.py", line 72, in <module>
    run_migrations_online()
  File "alembic/env.py", line 65, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/usr/local/lib/python2.7/site-packages/alembic/runtime/environment.py", line 797, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/local/lib/python2.7/site-packages/alembic/runtime/migration.py", line 312, in run_migrations
    step.migration_fn(**kw)
  File "/Users/fredrik/code/repos/ir-skalman/submodules/stalker/alembic/versions/f2005d1fbadc_added_projectclients.py", line 62, in upgrade
    use_alter=True
  File "<string>", line 8, in create_foreign_key
  File "<string>", line 3, in create_foreign_key
  File "/usr/local/lib/python2.7/site-packages/alembic/operations/ops.py", line 564, in create_foreign_key
    return operations.invoke(op)
  File "/usr/local/lib/python2.7/site-packages/alembic/operations/base.py", line 318, in invoke
    return fn(self, operation)
  File "/usr/local/lib/python2.7/site-packages/alembic/operations/toimpl.py", line 135, in create_constraint
    operation.to_constraint(operations.migration_context)
  File "/usr/local/lib/python2.7/site-packages/alembic/ddl/impl.py", line 180, in add_constraint
    self._exec(schema.AddConstraint(const))
  File "/usr/local/lib/python2.7/site-packages/alembic/ddl/impl.py", line 118, in _exec
    return conn.execute(construct, *multiparams, **params)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 968, in _execute_ddl
    compiled
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) constraint "xu" for relation "SimpleEntities" already exists
 [SQL: 'ALTER TABLE "SimpleEntities" ADD CONSTRAINT xu FOREIGN KEY(updated_by_id) REFERENCES "Users" (id)']

I also tried this with pg8000 (which I prefer because of the lack of dependencies):

$ alembic upgrade head

INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade 39d3c16ff005 -> 258985128aff, create EntityGroups table
INFO  [alembic.runtime.migration] Running upgrade 258985128aff -> 745b210e6907, fix non existing thumbnails
INFO  [alembic.runtime.migration] Running upgrade 745b210e6907 -> f2005d1fbadc, added ProjectClients
Traceback (most recent call last):
  File "/usr/local/bin/alembic", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/site-packages/alembic/config.py", line 479, in main
    CommandLine(prog=prog).main(argv=argv)
  File "/usr/local/lib/python2.7/site-packages/alembic/config.py", line 473, in main
    self.run_cmd(cfg, options)
  File "/usr/local/lib/python2.7/site-packages/alembic/config.py", line 456, in run_cmd
    **dict((k, getattr(options, k)) for k in kwarg)
  File "/usr/local/lib/python2.7/site-packages/alembic/command.py", line 174, in upgrade
    script.run_env()
  File "/usr/local/lib/python2.7/site-packages/alembic/script/base.py", line 397, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/usr/local/lib/python2.7/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/local/lib/python2.7/site-packages/alembic/util/compat.py", line 79, in load_module_py
    mod = imp.load_source(module_id, path, fp)
  File "alembic/env.py", line 72, in <module>
    run_migrations_online()
  File "alembic/env.py", line 65, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/usr/local/lib/python2.7/site-packages/alembic/runtime/environment.py", line 797, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/local/lib/python2.7/site-packages/alembic/runtime/migration.py", line 312, in run_migrations
    step.migration_fn(**kw)
  File "/Users/fredrik/code/repos/ir-skalman/submodules/stalker/alembic/versions/f2005d1fbadc_added_projectclients.py", line 62, in upgrade
    use_alter=True
  File "<string>", line 8, in create_foreign_key
  File "<string>", line 3, in create_foreign_key
  File "/usr/local/lib/python2.7/site-packages/alembic/operations/ops.py", line 564, in create_foreign_key
    return operations.invoke(op)
  File "/usr/local/lib/python2.7/site-packages/alembic/operations/base.py", line 318, in invoke
    return fn(self, operation)
  File "/usr/local/lib/python2.7/site-packages/alembic/operations/toimpl.py", line 135, in create_constraint
    operation.to_constraint(operations.migration_context)
  File "/usr/local/lib/python2.7/site-packages/alembic/ddl/impl.py", line 180, in add_constraint
    self._exec(schema.AddConstraint(const))
  File "/usr/local/lib/python2.7/site-packages/alembic/ddl/impl.py", line 118, in _exec
    return conn.execute(construct, *multiparams, **params)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 968, in _execute_ddl
    compiled
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/local/lib/python2.7/site-packages/pg8000/core.py", line 906, in execute
    self._c.execute(self, operation, args)
  File "/usr/local/lib/python2.7/site-packages/pg8000/core.py", line 2011, in execute
    self.handle_messages(cursor)
  File "/usr/local/lib/python2.7/site-packages/pg8000/core.py", line 2088, in handle_messages
    raise self.error
sqlalchemy.exc.ProgrammingError: (pg8000.core.ProgrammingError) (u'ERROR', u'42710', u'constraint "xu" for relation "SimpleEntities" already exists', u'src\\backend\\commands\\tablecmds.c', u'5621', u'ATExecAddConstraint', u'', u'') [SQL: u'ALTER TABLE "SimpleEntities" ADD CONSTRAINT xu FOREIGN KEY(updated_by_id) REFERENCES "Users" (id)']

Multiple repositories

I'm designing our pipeline to take advantage of multiple network shares, meaning multiple repositories. The reason for this is to store old versions of files and renderings on a cheaper and slower storage solution.

I basically need somewhere to define where this secondary network share is located and I want to specify this on the Project.

So, I have been considering creating a Link and store that on the Project.references, but that is not - per definition - a reference, really. I could also store this in the generic_text, which I am now using for arbitrary data stored in JSON format. However, I feel that this could perhaps be one of those things a lot of people do - and not just me.

What do you think?

Error in main Readme?

Hello,

Trying to setup Stalker db, it seems the save function is no more in DBSession ?

Is the right replacement .add() ?

Thanks

Cannot associate user with client on user creation

I'm having issues with connecting a user to a company upon user creation. This is how I create the user and associate it with a client, but this does not work:

user = User(
            name=user,
            login=username,
            email=email,
            password=password,
            company_id=client_id
           )
db.DBSession.add( user )
db.DBSession.commit()

However, this works:

user = User(
            name=user,
            login=username,
            email=email,
            password=password
           )
user.company_id = client_id
db.DBSession.add( user )
db.DBSession.commit()

Revision upgrade af869ddfdf9 -> 59bfe820c369 fails

From the current stalker revision, I performed a downgrade -1 without issues:

\\192.168.0.225\Pipeline\source\stalker [develop +3 ~8 -0 !]> alembic downgrade
-1
INFO  [alembic.migration] Context impl MySQLImpl.
INFO  [alembic.migration] Will assume non-transactional DDL.
C:\Python27\lib\site-packages\alembic-0.6.5-py2.7.egg\alembic\util.py:189: UserW
arning: Revision 182f44ce5f07 is present more than once
  warnings.warn(msg)
INFO  [alembic.migration] Running downgrade 59bfe820c369 -> af869ddfdf9, Added U
ser.efficiency column

But when I attempted to do an upgrade +1 back to the current revision I got this... (could it be MySQL specific?):

\\192.168.0.225\Pipeline\source\stalker [develop +4 ~8 -0 !]> alembic upgrade +1

INFO  [alembic.migration] Context impl MySQLImpl.
INFO  [alembic.migration] Will assume non-transactional DDL.
C:\Python27\lib\site-packages\alembic-0.6.5-py2.7.egg\alembic\util.py:189: UserW
arning: Revision 182f44ce5f07 is present more than once
  warnings.warn(msg)
INFO  [alembic.migration] Running upgrade af869ddfdf9 -> 59bfe820c369, Added Use
r.efficiency column
Traceback (most recent call last):
  File "C:\Python27\Scripts\alembic-script.py", line 9, in <module>
    load_entry_point('alembic==0.6.5', 'console_scripts', 'alembic')()
  File "C:\Python27\lib\site-packages\alembic-0.6.5-py2.7.egg\alembic\config.py"
, line 298, in main
    CommandLine(prog=prog).main(argv=argv)
  File "C:\Python27\lib\site-packages\alembic-0.6.5-py2.7.egg\alembic\config.py"
, line 293, in main
    self.run_cmd(cfg, options)
  File "C:\Python27\lib\site-packages\alembic-0.6.5-py2.7.egg\alembic\config.py"
, line 279, in run_cmd
    **dict((k, getattr(options, k)) for k in kwarg)
  File "C:\Python27\lib\site-packages\alembic-0.6.5-py2.7.egg\alembic\command.py
", line 125, in upgrade
    script.run_env()
  File "C:\Python27\lib\site-packages\alembic-0.6.5-py2.7.egg\alembic\script.py"
, line 203, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "C:\Python27\lib\site-packages\alembic-0.6.5-py2.7.egg\alembic\util.py",
line 212, in load_python_file
    module = load_module_py(module_id, path)
  File "C:\Python27\lib\site-packages\alembic-0.6.5-py2.7.egg\alembic\compat.py"
, line 58, in load_module_py
    mod = imp.load_source(module_id, path, fp)
  File "alembic\env.py", line 72, in <module>
    run_migrations_online()
  File "alembic\env.py", line 65, in run_migrations_online
    context.run_migrations()
  File "<string>", line 7, in run_migrations
  File "C:\Python27\lib\site-packages\alembic-0.6.5-py2.7.egg\alembic\environmen
t.py", line 688, in run_migrations
    self.get_context().run_migrations(**kw)
  File "C:\Python27\lib\site-packages\alembic-0.6.5-py2.7.egg\alembic\migration.
py", line 258, in run_migrations
    change(**kw)
  File "alembic\versions\59bfe820c369_resource_efficiency.py", line 20, in upgra
de
    op.execute('update "Users" set efficiency = 1.0')
  File "<string>", line 7, in execute
  File "C:\Python27\lib\site-packages\alembic-0.6.5-py2.7.egg\alembic\operations
.py", line 1024, in execute
    execution_options=execution_options)
  File "C:\Python27\lib\site-packages\alembic-0.6.5-py2.7.egg\alembic\ddl\impl.p
y", line 79, in execute
    self._exec(sql, execution_options)
  File "C:\Python27\lib\site-packages\alembic-0.6.5-py2.7.egg\alembic\ddl\impl.p
y", line 76, in _exec
    conn.execute(construct, *multiparams, **params)
  File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 720, in e
xecute
    return meth(self, multiparams, params)
  File "C:\Python27\lib\site-packages\sqlalchemy\sql\elements.py", line 317, in
_execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 817, in _
execute_clauseelement
    compiled_sql, distilled_params
  File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 947, in _
execute_context
    context)
  File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 1108, in
_handle_dbapi_exception
    exc_info
  File "C:\Python27\lib\site-packages\sqlalchemy\util\compat.py", line 185, in r
aise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 940, in _
execute_context
    context)
  File "C:\Python27\lib\site-packages\sqlalchemy\engine\default.py", line 435, i
n do_execute
    cursor.execute(statement, parameters)
  File "C:\Python27\lib\site-packages\MySQLdb\cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "C:\Python27\lib\site-packages\MySQLdb\connections.py", line 36, in defau
lterrorhandler
    raise errorclass, errorvalue
sqlalchemy.exc.ProgrammingError: (ProgrammingError) (1064, 'You have an error in
 your SQL syntax; check the manual that corresponds to your MySQL server version
 for the right syntax to use near \'"Users" set efficiency = 1.0\' at line 1') '
update "Users" set efficiency = 1.0' ()

Querying all projects, sorted by client name and then by project name

I'm having difficulties figuring out how I could be achieving the following... if you mind lending your ear and give me your thoughts, I'd be very grateful. :)

I have one Client object associated with each Project. Example:

Project A (Client B)
Project B (Client B)
Project C (Client A)

I can return all projects using this query:

Project.query.order_by(Project.name.asc()).all()

I'm now trying to figure out how I can query my projects so that they will be returned using a special sorting: first sorted by client name, then sorted by project name.

If I could achieve this, the query would return the projects in this order:

Project C (Client A)
Project A (Client B)
Project B (Client B)

I'm guessing I can't perform a simple "Stalker query" but instead would have to construct a custom SQL query?

First I tried this, but failed:

>>> Project.query.order_by(Project.clients[0].name.asc()).all()
TypeError: 'AssociationProxy' object does not support indexing

Currently what I do now is I just query using the above mentioned call, and then I sort "manually" using Python dictionaries. This is actually a lot slower than one would imagine and I've come to a conclusion that I need to construct a query from which I receive the objects in the proper order straight from the database rather than looping over dictionaries.

Postgres pubsub for near-realtime updates

So I've built this PySide/PyQt based production management UI which runs on top of Stalker and which we have running in Maya, Nuke and as standalone app. The UI runs on each client and needs to be "refreshed" in order to show changes performed by a different client. During certain events, the UI force-refreshes itself to keep it as up to date as possible with the database. An example of this is that I list all versions of a task, but this list doesn't automatically get updated unless the user e.g. updates a version, which is when I perform a query to show anything else that might be new in this list of versions.

The UI feels clunky because of this and is performing unnecessary queries against the database even though there may not be any changes in the database. Therefore I'm interested in looking into how I could use some sort of pubsub mechanism which would listen/notify of database changes for the current project and the UI's models would take appropriate actions based on the queue in the pubsub chat.

Postgres has built-in support for pubsub, which could be utilized for this particular thing. I believe it doesn't scale well if you have hundreds of clients (need to look into that) but since Stalker already has Postgres as a dependency, I thought the pubsub of Postgres might be worth looking into first (rather than going with a third-party pubsub solution such as Celery or RabbitMQ).

@eoyilmaz would you be interested to see Postgres pubsub functionality built into Stalker, or do you feel this will be out of scope for the project?
When running Stalker underneath PySide/PyQt, this feature would really take Stalker to the next level in my opinion.

I still haven't given this enough thought, but this is what I'm thinking initially that I would like to do:

  • One channel #stalker for notifications on global events which are not tied to a specific project. For example whenever a User, Status or Tag object is updated which is not necessarily tied to the project being browsed.
  • One channel per project #[Project.code] for notifications on creation/updating/deleting of primarily Project, Task, Version objects (but perhaps best way to implement this is to do it once in the SimpleEntity SOM?). I read somewhere that payloads are limited to 8000 bytes, so the notifications would be very simple, I'm thinking something like #proj1: UPDATE Version.id=20 to notify that the Version object with id=20 was updated.
  • Ability to open up custom channels for which you can perform custom NOTIFY/LISTEN. For example, I could use this to notify of a newer version of my UI tool. But also, you may wish to monitor e.g. all changes made to all Client objects, for some custom reason, regardless of which project you are browsing. In fact, I wish to do exactly this.
  • New Stalker submodule handling connecting and disconnecting to channels as well as notifying and listening to a particular channel. This would include making it easy to disconnect from a channel e.g. when the UI has been unresponsive for longer periods of time, or when changing from one project to another (using a frontend such as our UI).
  • I'm thinking that any pubsub channel must be explicitly created before any SOM starts notifying into it. There are many scenarios when pubsub functionality is not wanted (when running Stalker on a single client for example). But once the channel exists, the SOM will detect this and notify into it, like described previously.
  • Ability to add custom callback functions based different LISTEN events, so that e.g. our UI can update its model data based on the pubsub queue.

Also:

  • It would be interesting to explore the possibility to run the pubsub on a separate thread, as well as performing queries on a separate thread to avoid e.g. an UI to freeze during a query.

My hopes are that having Stalker underneath a Model-View based PySide UI using pubsub will make the UI feel like it's streaming Stalker data in near real-time, removing the feel of "disconnect" with the surrounding world, alleviating the need for "manual refreshes/querying". Instead, I can make our UI query the objects only when they are being updated, and ideally without a UI freeze.

Here are some related code examples which I'll be looking at:

Please let me know what your thoughts are about this are.

TypeError: Project.code cannot be None

Hello,
your library appears to be great and i want to test it.
Base on the official doc, i try this:

import os
import datetime
from stalker import db, User, Department, Status, StatusList, Project, Repository, ImageFormat

db_file = "/tmp/studio.db"
if os.path.exists( db_file ):
    os.unlink( db_file )
db.setup({"sqlalchemy.url": "sqlite:///%s" % db_file })


myUser = User(
    name="Erkan Ozgur Yilmaz",
    login="eoyilmaz",
    email="[email protected]",
    password="secret",
    description="This is me"
)

tds_department = Department(
    name="TDs",
    description="This is the TDs department"
)

tds_department.users.append(myUser)

db.session.DBSession.add(myUser)
db.session.DBSession.add(tds_department)
db.session.DBSession.commit()


# lets create a couple of generic Statuses
status_waiting = Status(name="Waiting To Start", code="WTS")
status_wip = Status(name="Work in Progress", code="WIP")
status_pendrev = Status(name="Pending Review", code="PREV")
status_approved = Status(name="Approved", code="APP")
status_complete = Status(name="Complete", code="CMPLT")
status_stopped = Status(name="Stopped", code="STOP")

# a status list which is suitable for Project instances
project_statuses = StatusList(
    name="Project Status List",
    statuses=[status_waiting,
              status_wip,
              status_stopped,
              status_complete],
    target_entity_type=Project
)

# and the repository itself
commercial_repo = Repository( name="Commercial Repository" )

new_project = Project(
    name="Fancy Commercial",
    status_list=project_statuses,
    repository=commercial_repo
)

new_project.description = """The commercial is about this fancy product. The
                             client want us to have a shiny look with their
                             product bla bla bla..."""
new_project.image_format = ImageFormat(name="HD 1080", width=1920, height=1080)
new_project.fps = 25
new_project.end = datetime.date(2011, 2, 15)
new_project.lead = myUser

db.session.DBSession.add(new_project)
db.session.DBSession.commit()

First thing is that 'db.session.commit()' does not work, i've to use 'db.session.DBSession.commit()'.

Second, when i execute the script, i've this error:

DEBUG:stalker.db:settings: {'sqlalchemy.url': 'sqlite:////tmp/studio.db'}
DEBUG:stalker.db:engine: Engine(sqlite:////tmp/studio.db)
DEBUG:stalker.db:creating the tables
DEBUG:stalker.models.auth:name out: eoyilmaz
DEBUG:stalker.models.mixins:validating code value of: WTS
DEBUG:stalker.models.mixins:validating code value of: WIP
DEBUG:stalker.models.mixins:validating code value of: PREV
DEBUG:stalker.models.mixins:validating code value of: APP
DEBUG:stalker.models.mixins:validating code value of: CMPLT
DEBUG:stalker.models.mixins:validating code value of: STOP
DEBUG:stalker.models.mixins:validating code value of: None
Traceback (most recent call last):
  File "tuto.py", line 57, in <module>
    repository=commercial_repo
  File "<string>", line 4, in __init__
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sqlalchemy/orm/state.py", line 196, in _initialize_instance
    return manager.original_init(*mixed[1:], **kwargs)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/stalker/models/project.py", line 236, in __init__
    self.code = code
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 220, in __set__
    instance_dict(instance), value, None)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 680, in set
    value, old, initiator)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sqlalchemy/orm/attributes.py", line 698, in fire_replace_event
    value = fn(state, value, previous, initiator or self._replace_token)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sqlalchemy/orm/util.py", line 107, in set_
    return validator(state.obj(), key, value)
  File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/stalker/models/mixins.py", line 920, in _validate_code
    raise TypeError("%s.code cannot be None" % self.__class__.__name__)
TypeError: Project.code cannot be None

Modifying User.companies results in AssertionError

I'm creating a user with some companies:

c1 = Client(name='Company X')
c2 = Client(name='Company Y')
c3 = Client(name='Company Z')

user = User(
        name='Fredrik',
        login='fredrik',
        email='[email protected]',
        password='pass',
        companies=[c1, c2, c3]
        ) 

db.DBSession.add( user )
db.DBSession.commit()

Then I modify the companies, effectively removing the old ones:

c1 = Client(name='New Company X')
c3 = Client(name='New Company Y')
c2 = Client(name='New Company Z')

user = User.query.filter_by(name='Fredrik').first()
user.companies = [c1, c2, c3]

db.DBSession.commit() # <--- triggers the error

And this causes an error:

AssertionError: Dependency rule tried to blank-out primary key column 'Client_Users.uid' on instance '<ClientUser at 0x58f60f0>'

Do you see this on your end too?

Delete a project with tasks and versions

How should I go about deleting a project from the db with all its tasks and all their versions?

It would've been nice to cascade the Project delete, but I don't want User objects, Status objects etc to get deleted which are associated with the project and the versions etc.

I was thinking I could achieve what I want to do in three steps:

  1. Delete all Version instances which are attached to a Task instance in Project.tasks
  2. Delete all Task instances in Project.tasks
  3. Delete the Project instance itself

@eoyilmaz would you recommend some other approach?

Error on "alembic upgrade head"

I ran alembic.exe upgrade head from my Python 3.5 installation on my Windows machine and I got this error:

INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade f2005d1fbadc -> 92257ba439e1, Budget is now statusable
INFO  [alembic.runtime.migration] Running upgrade 92257ba439e1 -> d8421de6a206, Added Project_Users.rate
INFO  [alembic.runtime.migration] Running upgrade d8421de6a206 -> ea28a39ba3f5, Added Invoices table
INFO  [alembic.runtime.migration] Running upgrade ea28a39ba3f5 -> 255ee1f9c7b3, Added Payments table
INFO  [alembic.runtime.migration] Running upgrade 255ee1f9c7b3 -> f16651477e64, Added AuthenticationLog class
INFO  [alembic.runtime.migration] Running upgrade f16651477e64 -> a9319b19f7be, added shot.fps
INFO  [alembic.runtime.migration] Running upgrade a9319b19f7be -> 0063f547dc2e, updated version_inputs table
INFO  [alembic.runtime.migration] Running upgrade 0063f547dc2e -> c5607b4cfb0a, Added support for time zones
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] upgrading table: AuthenticationLogs
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: date
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] done upgrading table: AuthenticationLogs
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] upgrading table: SimpleEntities
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: date_created
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: date_updated
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] done upgrading table: SimpleEntities
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] upgrading table: Tasks
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: computed_start
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: computed_end
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: start
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: end
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] done upgrading table: Tasks
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] upgrading table: Vacations
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: computed_start
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: computed_end
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: start
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: end
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] done upgrading table: Vacations
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] upgrading table: Projects
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: computed_start
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: computed_end
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: start
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: end
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] done upgrading table: Projects
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] upgrading table: TimeLogs
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: computed_start
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: computed_end
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: start
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: end
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] done upgrading table: TimeLogs
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] upgrading table: Studios
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: computed_start
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: computed_end
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: start
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: end
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: scheduling_started_at
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] altering column: last_scheduled_at
INFO  [c5607b4cfb0a_added_support_for_time_zones_py] done upgrading table: Studios
INFO  [alembic.runtime.migration] Running upgrade c5607b4cfb0a -> 31b1e22b455e, Added exclude and check constraints to TimeLogs table
INFO  [31b1e22b455e_added_exclude_and_check_constraints_to__py] Removing duplicate TimeLog entries
INFO  [31b1e22b455e_added_exclude_and_check_constraints_to__py] Removing contained TimeLog entries (TimeLogs surrounded by other TimeLogs
INFO  [31b1e22b455e_added_exclude_and_check_constraints_to__py] Trimming residual overlapping TimeLog.end values
INFO  [31b1e22b455e_added_exclude_and_check_constraints_to__py] Trimming residual overlapping TimeLog.start values
INFO  [31b1e22b455e_added_exclude_and_check_constraints_to__py] Adding CheckConstraint(end > start) to TimeLogs table
INFO  [31b1e22b455e_added_exclude_and_check_constraints_to__py] Adding ExcludeConstraint to TimeLogs table
DEBUG [stalker.models.task] add_exclude_constraint is Running!
DEBUG [stalker.models.task] running "btree_gist" extension creation!
DEBUG [stalker.models.task] add_exclude_constraint: (psycopg2.ProgrammingError) permission denied to create extension "btree_gist"
HINT:  Must be superuser to create this extension.
 [SQL: 'CREATE EXTENSION btree_gist;']
DEBUG [stalker.models.task] creating ts_to_box function!
Traceback (most recent call last):
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\sqlalchemy\engine\base.py", line 1182, in _execute_context
    context)
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\sqlalchemy\engine\default.py", line 469, in do_execute
    cursor.execute(statement, parameters)
psycopg2.InternalError: current transaction is aborted, commands ignored until end of transaction block


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\users\iruser\skalman\condaenvs\skalman_py35\Scripts\alembic.exe\__main__.py", line 9, in <module>
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\alembic\config.py", line 479, in main
    CommandLine(prog=prog).main(argv=argv)
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\alembic\config.py", line 473, in main
    self.run_cmd(cfg, options)
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\alembic\config.py", line 456, in run_cmd
    **dict((k, getattr(options, k)) for k in kwarg)
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\alembic\command.py", line 174, in upgrade
    script.run_env()
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\alembic\script\base.py", line 407, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\alembic\util\pyfiles.py", line 93, in load_python_file
    module = load_module_py(module_id, path)
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\alembic\util\compat.py", line 68, in load_module_py
    module_id, path).load_module(module_id)
  File "<frozen importlib._bootstrap_external>", line 396, in _check_name_wrapper
  File "<frozen importlib._bootstrap_external>", line 817, in load_module
  File "<frozen importlib._bootstrap_external>", line 676, in load_module
  File "<frozen importlib._bootstrap>", line 268, in _load_module_shim
  File "<frozen importlib._bootstrap>", line 693, in _load
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 673, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "alembic\env.py", line 72, in <module>
    run_migrations_online()
  File "alembic\env.py", line 65, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\alembic\runtime\environment.py", line 797, in run_migrations
    self.get_context().run_migrations(**kw)
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\alembic\runtime\migration.py", line 312, in run_migrations
    step.migration_fn(**kw)
  File "C:\Users\iruser\code\repos\stalker\alembic\versions\31b1e22b455e_added_exclude_and_check_constraints_to_.py", line 123, in upgrade
    add_exclude_constraint(TimeLog.__table__, conn)
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\stalker\models\task.py", line 3189, in add_exclude_constraint
    ts_to_box.execute(bind=connection)
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\sqlalchemy\sql\ddl.py", line 96, in execute
    return bind.execute(self.against(target))
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\sqlalchemy\engine\base.py", line 945, in execute
    return meth(self, multiparams, params)
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\sqlalchemy\sql\ddl.py", line 68, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\sqlalchemy\engine\base.py", line 1002, in _execute_ddl
    compiled
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\sqlalchemy\engine\base.py", line 1189, in _execute_context
    context)
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\sqlalchemy\engine\base.py", line 1393, in _handle_dbapi_exception
    exc_info
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\sqlalchemy\util\compat.py", line 202, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\sqlalchemy\util\compat.py", line 185, in reraise
    raise value.with_traceback(tb)
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\sqlalchemy\engine\base.py", line 1182, in _execute_context
    context)
  File "c:\users\iruser\skalman\condaenvs\skalman_py35\lib\site-packages\sqlalchemy\engine\default.py", line 469, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.InternalError: (psycopg2.InternalError) current transaction is aborted, commands ignored until end of transaction block
 [SQL: "CREATE FUNCTION ts_to_box(TIMESTAMPTZ, TIMESTAMPTZ)\nRETURNS BOX\nAS\n$$\n    SELECT  BOX(\n      POINT(DATE_PART('epoch', $1), 0),\n      POINT(DATE_PART('epoch', $2 - interval '1 minute'), 1)\n    )\n$$\nLANGUAGE 'sql'\nIMMUTABLE;\n"]

"pip install" fails when stalker exists as dependency

I've created a wheel, which has stalker as a dependency. When I install this wheel, it always fails like this:

$ pip install [REDACTED]/skalman-2.2.21-py2.py3-none-any.whl
Collecting skalman==2.2.21 from [REDACTED]/skalman-2.2.21-py2.py3-none-any.whl
  Downloading [REDACTED]/skalman-2.2.21-py2.py3-none-any.whl (5.0MB)
    100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 5.0MB 119kB/s
Collecting six==1.10.0 (from skalman==2.2.21)
  Downloading six-1.10.0-py2.py3-none-any.whl
Collecting PyYAML==3.12 (from skalman==2.2.21)
  Downloading PyYAML-3.12.tar.gz (253kB)
    100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 256kB 1.3MB/s
Collecting alembic==0.8.8 (from skalman==2.2.21)
  Downloading alembic-0.8.8.tar.gz (970kB)
    100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 972kB 566kB/s
Collecting packaging (from skalman==2.2.21)
  Downloading packaging-16.8-py2.py3-none-any.whl
Collecting Mako==1.0.6 (from skalman==2.2.21)
  Downloading Mako-1.0.6.tar.gz (575kB)
    100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 583kB 910kB/s
Collecting Jinja2==2.8 (from skalman==2.2.21)
  Downloading Jinja2-2.8-py2.py3-none-any.whl (263kB)
    100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 266kB 2.3MB/s
Collecting pg8000==1.10.6 (from skalman==2.2.21)
  Downloading pg8000-1.10.6-py2.py3-none-any.whl
Collecting Qt.py==1.0.0b3 (from skalman==2.2.21)
  Downloading Qt.py-1.0.0b3-py2.py3-none-any.whl
Collecting SQLAlchemy==1.1.4 (from skalman==2.2.21)
  Downloading SQLAlchemy-1.1.4.tar.gz (5.1MB)
    100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 5.1MB 116kB/s
Collecting colorlog==2.10.0 (from skalman==2.2.21)
  Downloading colorlog-2.10.0-py2.py3-none-any.whl
Collecting stalker==0.2.18 (from skalman==2.2.21)
  Downloading stalker-0.2.18.zip (803kB)
    100% |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 808kB 739kB/s
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-zu_evvg4/stalker/setup.py", line 5, in <module>
        import stalker
      File "/tmp/pip-build-zu_evvg4/stalker/stalker/__init__.py", line 39, in <module>
        from stalker.models.auth import (Group, Permission, User, LocalSession, Role,
      File "/tmp/pip-build-zu_evvg4/stalker/stalker/models/auth.py", line 27, in <module>
        from sqlalchemy import (Table, Column, Integer, ForeignKey, String, DateTime,
    ModuleNotFoundError: No module named 'sqlalchemy'

This is in my wheel's setup.py:

install_requires=[
        'alembic==0.8.8', 'colorlog==2.10.0', 'Jinja2==2.8', 'Mako==1.0.6',
        'MarkupSafe==0.23', 'pg8000==1.10.6', 'pyseq==0.5.0',
        'python-editor==1.0.1', 'PyYAML==3.12', 'Qt.py==1.0.0b3',
        'requests==2.13.0', 'six==1.10.0', 'SQLAlchemy==1.1.4', 'pytz==2016.10',
        'stalker==0.2.18', 'setuptools', 'packaging'
    ],

So it seems the import in that __init__.py is causing the issue. Perhaps this should be put inside of a try/except clause - or have some detection of it being executed by pip?

The workaround is to first pip install SQLAlchemy and then install my wheel.

config.py datetime module error

After I create a config.py in the STALKER_PATH I got this error while importing stalker:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\virtualenv\stalker\lib\site-packages\stalker\__init__.py", line 30, in <module>
    from stalker.config import defaults
  File "D:\virtualenv\stalker\lib\site-packages\stalker\config.py", line 540, in <module>
    defaults = Config()
  File "D:\virtualenv\stalker\lib\site-packages\stalker\config.py", line 471, in __init__
    self._parse_settings()
  File "D:\virtualenv\stalker\lib\site-packages\stalker\config.py", line 507, in _parse_settings
    execfile(resolved_path, self.user_config)
  File "D:\STALKER_MOD\config.py", line 173, in <module>
    timing_resolution = datetime.timedelta(hours=1)
NameError: name 'datetime' is not defined

solved it by importing the datetime module in the beginning of the file, but maybe this is not a expected behavior?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.