redshodan / unsonic Goto Github PK

• Add last.fm API support
• Store credentials in the DB
• have a command to manage the creds #27
• Implement the subsonic calls that need it
• this allows for #8

I'm sure I can come up with uses... But for now, backlog the place holder

• Watch for stupid clients pushing the same scrobble multiple times too fast.

• If no config file on the cmdline, look: /etc/unsonic.ini, INSTALL/etc/unsonic.ini
• Package up production.ini as unsonic/etc/unsonic.ini.in
• Add systemd file: #23
• Install script that can be run to fully setup unsonic

Make it work.

Add support for multiple languages. Yes, I know this should be done sooner than later, but..

• intl protocol strings
• select lastfm language

Have Unsonic front for other music services, like Google Music. Expose the gmusic library as part of the unsonic library for unsonic clients to use.

https://unofficial-google-music-api.readthedocs.io/en/latest/

2019-03-31 10:39:11,731 ERROR [unsonic][waitress] Error talking to LastFM: Invalid session key - Please re-authenticate

Expected behavior

For everything to just work. :)

Actual behavior

It did not.

Steps to reproduce the behavior

make MUSIC_DIR=/music docker-postgres

Steps to fix

see comment below

• Code of conduct
• Contributing: CONTRIBUTING.md
• Issue or pull request template

General outline:

• general use. Tags based.
• pip install
• config
• syncing
• running server
• TLS nginx
• Dev howto

Doc stuff:

• Filter README.md for:
• man page
• html
• setup.py README

Have a startup password to (de)-encrypt the secrets in the database, like credentials for lastfm. Ask for the password on startup, keep decrypted data in memory only.

unsonic.nicfit.net    | 2017-05-04 01:02:36,794 INFO  [unsonic.views.rest.scrobble][waitress] Scrobbling Oh
unsonic.nicfit.net    | 2017-05-04 01:02:37,394 INFO  [wsgi][waitress] 74.220.34.132 - - [04/May/2017:01:02:37 +0000] "GET /rest/stream.view?u=nicfit&s=h6km5imj47943sg2392qm8aap5&t=81d52df812f89b1cfceac5cfade91790&v=1.2.0&c=DSub&id=tr-12788&maxBitRate=196 HTTP/1.1" 200 6371328 "-" "DSub"
PostgreSQL-unsonic.nicfit.net | ERROR:  duplicate key value violates unique constraint "pk_un_playcounts"
PostgreSQL-unsonic.nicfit.net | DETAIL:  Key (track_id, user_id)=(12787, 2) already exists.
PostgreSQL-unsonic.nicfit.net | STATEMENT:  INSERT INTO un_playcounts (track_id, user_id, count) VALUES (12787, 2, 1)
unsonic.nicfit.net    | 2017-05-04 01:02:37,830 INFO  [wsgi][waitress] 74.220.34.132 - - [04/May/2017:01:02:35 +0000] "GET /rest/scrobble.view?u=nicfit&s=h6km5imj47943sg2392qm8aap5&t=81d52df812f89b1cfceac5cfade91790&v=1.2.0&c=DSub&id=tr-12787&submission=false HTTP/1.1" 200 139 "-" "DSub"
unsonic.nicfit.net    | 2017-05-04 01:02:39,556 INFO  [wsgi][waitress] 74.220.34.132 - - [04/May/2017:01:02:39 +0000] "GET /rest/savePlayQueue.view?u=nicfit&s=h6km5imj47943sg2392qm8aap5&t=81d52df812f89b1cfceac5cfade91790&v=1.2.0&c=DSub&id=tr-12781&id=tr-12782&id=tr-12783&id=tr-12784&id=tr-12785&id=tr-12786&id=tr-12787&id=tr-12788&id=tr-12789&id=tr-12790&id=tr-32976&current=tr-12788&position=0 HTTP/1.1" 200 139 "-" "DSub"

The 1.16 version added RadioStations. Looks like just 4 new endpoints were added.

• getInternetRadioStations
• createInternetRadioStation
• updateInternetRadioStation
• deleteInternetRadioStation

The 1.15 API only adds two calls for music scanning. Implement those interfaces and scanning in the web server process in a different thread.

Hi, users are unable to run Unsonic due to dependency conflict with pastedeploy package. As shown in the following full dependency graph of Unsonic, Unsonic requires pastedeploy==1.5.2，while plaster-pastedeploy * requires pastedeploy>=2.0.

According to pip’s “first found wins” installation strategy, pastedeploy 1.5.2 is the actually installed version. However, pastedeploy 1.5.2 does not satisfy pastedeploy>=2.0.

Dependency tree-----------

unsonic - 0.1.3
| +- alembic(install version:0.9.7 version range:==0.9.7)
| | +- mako(install version:1.1.2 version range:*)
| | | +- MarkupSafe(install version:2.0.0a1 version range:>=0.9.2)
| | +- python-dateutil(install version:2.8.1 version range:*)
| | +- python-editor(install version:1.0.4 version range:>=0.3)
| | +- sqlalchemy(install version:1.2.2 version range:>=0.7.6)
| +- mishmash(install version:0.3b5 version range:==0.3b5)
| +- paste(install version:2.0.3 version range:==2.0.3)
| | +- six(install version:1.14.0 version range:>=1.4.0)
| +- pastedeploy(install version:1.5.2 version range:==1.5.2)
| +- psycopg2(install version:2.7.3.2 version range:==2.7.3.2)
| +- pyramid(install version:1.9.1 version range:==1.9.1)
| +- pyramid-debugtoolbar(install version:4.3 version range:==4.3)
| | +- pygments(install version:2.6.1 version range:*)
| | +- pyramid(install version:1.9.1 version range:>=1.4)
| | +- pyramid-mako(install version:1.1.0 version range:>=0.3.1)
| | | +- Mako(install version:1.1.2 version range:>=1.1.0)
| | | | +- MarkupSafe(install version:2.0.0a1 version range:>=0.9.2)
| | | +- pyramid(install version:1.9.1 version range:*)
| | +- repoze-lru(install version:0.7 version range:*)
| +- pyramid-tm(install version:2.2 version range:==2.2)
| | +- pyramid (install version:1.10.4 version range:>=1.5)
| | | +- hupper(install version:1.10.2 version range:>=1.5)
| | | +- plaster(install version:1.0 version range:*)
| | | | +- setuptools(install version:46.1.3 version range:*)
| | | +- plaster-pastedeploy(install version:0.7 version range:*)
| | | | +- pastedeploy(install version:2.1.0 version range:>=2.0)
| | | | +- plaster(install version:1.0 version range:>=0.5)
| | | +- setuptools(install version:46.1.3 version range:*)
| | | +- translationstring(install version:1.3 version range:>=0.4)
| | | +- venusian(install version:3.0.0 version range:>=1.0)
| | | +- webob(install version:1.8.6 version range:>=1.8.3)
| | | +- zope-deprecation(install version:4.4.0 version range:>=3.5.0)
| | | | +- setuptools(install version:46.1.3 version range:*)
| | | +- zope-interface(install version:5.1.0 version range:>=3.8.0)
| | +- transaction (install version:3.0.0 version range:>=2.0)
| | | +- zope.interface(install version:5.1.0 version range:*)
| +- recommonmark(install version:0.4.0 version range:==0.4.0)
| | +- commonmark(install version:0.5.4 version range:<=0.5.4)
| | +- docutils(install version:0.16 version range:>=0.11)
| +- sphinx(install version:1.6.6 version range:==1.6.6)
| +- sqlalchemy(install version:1.2.2 version range:==1.2.2)
| +- transaction(install version:2.1.2 version range:==2.1.2)
| | +- zope-interface(install version:5.1.0 version range:*)
| +- waitress(install version:1.1.0 version range:==1.1.0)
| +- xmltodict(install version:0.11.0 version range:==0.11.0)
| +- zope-sqlalchemy(install version:0.7.7 version range:==0.7.7)
| | +- setuptools(install version:46.1.3 version range:*)
| | +- sqlalchemy(install version:1.2.2 version range:>=0.5.1)
| | +- transaction(install version:2.1.2 version range:*)
| | | +- zope-interface(install version:5.1.0 version range:*)
| | +- zope-interface(install version:5.1.0 version range:>=3.6.0)

Thanks for your help.
Best,
Neolith

Add a page that says how to install jamstash or another web client that is shown when no web client is configured or is not found.

'''
unsonic.nicfit.net | 2019-05-22 07:38:11,019 DEBUG [txn.139753145244392][waitress] new transaction
unsonic.nicfit.net | Traceback (most recent call last):
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/unsonic/views/rest/init.py", line 80, in call
unsonic.nicfit.net | return self.handleReq(session)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/unsonic/views/rest/getrandomsongs.py", line 48, in handleReq
unsonic.nicfit.net | join(Album).filter(Album.release_date >= fy and
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 3054, in bool
unsonic.nicfit.net | raise TypeError("Boolean value of this clause is not defined")
unsonic.nicfit.net | TypeError: Boolean value of this clause is not defined
unsonic.nicfit.net | 2019-05-22 07:38:11,025 DEBUG [txn.139753145244392][waitress] abort
unsonic.nicfit.net | 2019-05-22 07:38:11,028 ERROR [waitress][waitress] Exception when serving /rest/getRandomSongs.view
unsonic.nicfit.net | Traceback (most recent call last):
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid/tweens.py", line 13, in error_handler
unsonic.nicfit.net | response = request.invoke_exception_view(exc_info)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid/view.py", line 769, in invoke_exception_view
unsonic.nicfit.net | raise HTTPNotFound
unsonic.nicfit.net | pyramid.httpexceptions.HTTPNotFound: The resource could not be found.
unsonic.nicfit.net |
unsonic.nicfit.net | During handling of the above exception, another exception occurred:
unsonic.nicfit.net |
unsonic.nicfit.net | Traceback (most recent call last):
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/waitress/channel.py", line 336, in service
unsonic.nicfit.net | task.service()
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/waitress/task.py", line 175, in service
unsonic.nicfit.net | self.execute()
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/waitress/task.py", line 452, in execute
unsonic.nicfit.net | app_iter = self.channel.server.application(env, start_response)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/paste/translogger.py", line 69, in call
unsonic.nicfit.net | return self.application(environ, replacement_start_response)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid/router.py", line 270, in call
unsonic.nicfit.net | response = self.execution_policy(environ, self)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid/router.py", line 279, in default_execution_policy
unsonic.nicfit.net | return request.invoke_exception_view(reraise=True)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid/view.py", line 768, in invoke_exception_view
unsonic.nicfit.net | reraise(*exc_info)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid/compat.py", line 179, in reraise
unsonic.nicfit.net | raise value
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid/router.py", line 277, in default_execution_policy
unsonic.nicfit.net | return router.invoke_request(request)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid/router.py", line 249, in invoke_request
unsonic.nicfit.net | response = handle_request(request)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid_tm/init.py", line 171, in tm_tween
unsonic.nicfit.net | reraise(*exc_info)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid_tm/compat.py", line 36, in reraise
unsonic.nicfit.net | raise value
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid_tm/init.py", line 136, in tm_tween
unsonic.nicfit.net | response = handler(request)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid/tweens.py", line 43, in excview_tween
unsonic.nicfit.net | response = _error_handler(request, exc)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid/tweens.py", line 17, in _error_handler
unsonic.nicfit.net | reraise(*exc_info)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid/compat.py", line 179, in reraise
unsonic.nicfit.net | raise value
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid/tweens.py", line 41, in excview_tween
unsonic.nicfit.net | response = handler(request)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid/router.py", line 148, in handle_request
unsonic.nicfit.net | registry, request, context, context_iface, view_name
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid/view.py", line 657, in _call_view
unsonic.nicfit.net | response = view_callable(context, request)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid/viewderivers.py", line 325, in secured_view
unsonic.nicfit.net | return view(context, request)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid/viewderivers.py", line 401, in viewresult_to_response
unsonic.nicfit.net | result = view(context, request)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/pyramid/viewderivers.py", line 130, in _class_view
unsonic.nicfit.net | response = inst()
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/unsonic/views/rest/init.py", line 80, in call
unsonic.nicfit.net | return self.handleReq(session)
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/unsonic/views/rest/getrandomsongs.py", line 48, in handleReq
unsonic.nicfit.net | join(Album).filter(Album.release_date >= fy and
unsonic.nicfit.net | File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 3054, in bool
unsonic.nicfit.net | raise TypeError("Boolean value of this clause is not defined")
unsonic.nicfit.net | TypeError: Boolean value of this clause is not defined
unsonic.nicfit.net | 2019-05-22 07:38:11,033 DEBUG [txn.139753147366120][waitress] new transaction

'''

• Create service file
• Package service file

oauth support will be great ...

When coming online Dsub...

2018-03-26 22:02:17,505 INFO  [wsgi][waitress] [email protected] [26/Mar/2018:22:02:17 -0600] "GET /rest/search2.view?u=nicfit&&v=1.2.0&c=DSub&query=artist%3A%22The+Freeze%22+AND+title%3A%22Bloodlights%22&artistCount=0&albumCount=0&songCount=1 HTTP/1.1" 200 174 "-" "DSub"

One per song played while offline.

With nicfit/MishMash#286, compare LastFM artist, album, tracks based on slugified versions of the names.

Expected behavior

$ unsonic -c ... serve 

Runs the Unsonic API server.

Actual behavior

Fails with exception:

$ bin/unsonic -c docker/config.ini serve
2018-01-23 12:52:30,396 ERROR [mishmash][MainThread] 'Serve' object has no attribute 'config'
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/mishmash/__main__.py", line 49, in main
    retval = args.command_func(args, args.config) or 0
  File "/Users/bschumac/src/repos/unsonic/unsonic/commands/serve.py", line 26, in run
    super()._run(args)
  File "/Users/bschumac/src/repos/unsonic/unsonic/commands/__init__.py", line 21, in _run
    initAlembic(self.config.get("mishmash", "sqlalchemy.url"))
AttributeError: 'Serve' object has no attribute 'config'
General error:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/mishmash/__main__.py", line 49, in main
    retval = args.command_func(args, args.config) or 0
  File "/Users/bschumac/src/repos/unsonic/unsonic/commands/serve.py", line 26, in run
    super()._run(args)
  File "/Users/bschumac/src/repos/unsonic/unsonic/commands/__init__.py", line 21, in _run
    initAlembic(self.config.get("mishmash", "sqlalchemy.url"))
AttributeError: 'Serve' object has no attribute 'config'
2018-01-23 12:52:30,397 DEBUG [unsonic][MainThread] Application._atexit

This appears to be related to change ef3680a.

Steps to reproduce the behavior

Start Unsonic with serve.

The startup takes longer than I'd like. It's certainly mostly work around the DB, but do a profile to check if its not something stupid.

Click on artist in the library and dsub makes a request for artist bio info and shows an error dialog
"resource not found". The request:

unsonic.nicfit.net | 2017-03-28 01:31:17,388 INFO [wsgi][waitress] 192.168.1.114 - - [28/Mar/2017:01:31:17 +0000] "GET /rest/getArtistInfo2.view?u=nicfit&s=XXXXXXXXXXXXX&t=XXXXXXXXXXXXXX&v=1.2.0&c=DSub&id=ar-9&includeNotPresent=true HTTP/1.1" 404 77 "-" "DSub"

• Create a pip package for jamstash that works with unsonic
• Use the static assets in pyramid to serve it up from installed source
• Make it optional so it doesn't break anything if not installed
• Make sure the Jamstash license is kosher
• Pre-populate the cfg bits for Jamstash
• don't land on the settings page once it's pre-populated

Mishmash issues required:

• DB column size limits: nicfit/MishMash#209
• New mishmash release

Error:
Missing a required parameter: Invalid ID

Make use of mishmash's sync monitor mode to watch for changes to the music directories and sync back up. Use a subprocess to run mishmash directly.

My commit was to fix this, it doesn't.
Although the files are included in the dist now.

Unsonic-sqlite      | General error:
Unsonic-sqlite      | Traceback (most recent call last):
Unsonic-sqlite      |   File "/unsonic/lib/python3.6/site-packages/mishmash/__main__.py", line 46, in main
Unsonic-sqlite      |     retval = args.command_func(args, args.config) or 0
Unsonic-sqlite      |   File "/unsonic/lib/python3.6/site-packages/mishmash/core.py", line 16, in run
Unsonic-sqlite      |     retval = super().run(args)
Unsonic-sqlite      |   File "/unsonic/lib/python3.6/site-packages/nicfit/command.py", line 49, in run
Unsonic-sqlite      |     return self._run()
Unsonic-sqlite      |   File "/unsonic/lib/python3.6/site-packages/unsonic/commands/sync.py", line 13, in newrun
Unsonic-sqlite      |     initAlembic()
Unsonic-sqlite      |   File "/unsonic/lib/python3.6/site-packages/unsonic/models.py", line 313, in initAlembic
Unsonic-sqlite      |     command.upgrade(alembic_cfg, "head")
Unsonic-sqlite      |   File "/unsonic/lib/python3.6/site-packages/alembic/command.py", line 254, in upgrade
Unsonic-sqlite      |     script.run_env()
Unsonic-sqlite      |   File "/unsonic/lib/python3.6/site-packages/alembic/script/base.py", line 416, in run_env
Unsonic-sqlite      |     util.load_python_file(self.dir, 'env.py')
Unsonic-sqlite      |   File "/unsonic/lib/python3.6/site-packages/alembic/util/pyfiles.py", line 99, in load_python_file
Unsonic-sqlite      |     raise ImportError("Can't find Python file %s" % path)
Unsonic-sqlite      | ImportError: Can't find Python file /unsonic/lib/python3.6/site-packages/unsonic/alembic/env.py

Polish up and publish the docker images.

Looks like the query syntax is a lot more capable than raw substring matching. Examples:

• For autocomplete as a user types: i*, in*, int*
• artist:"END TIMES" AND title:"Leviathan"

First run through it mostly works, manual run.

• requires change to mishmash, nicfit/MishMash#260.
• adjust install command to do launchd on osx

The unsonic the config, web, and models modules do bad things to the configuration. Between the needs of Pyramid, the test framework and mishmash, that code has gotten very ugly. It needs to be fixed up into a single interface that works for everything.

• Generic (totally not unosnic-only) Config class derived from base ConfigParser. Follow the ConfigParser API as much as possible for compat with other things.
• Configured the Config class to know about:
  • default .ini config file to read in first
  • database table to key/pairs to read from
  • environment variables, named after key name with app prefix
  • user ~/. config.ini
  • system /etc/*/config.ini

• Have a server side config option to choose how artists are sorted: via artist or album_artist.
• If possible, add some way to toggle that via the client. Most likely require an extension to the protocol and client mods.

• Allow for:
• adding values
• deleting values
• modifying values
• validate global/user settings. check that key name is used in the right context