ibi-group / datatools-server Goto Github PK

View Code? Open in Web Editor NEW

49.0 49.0 49.0 13.85 MB

Server for IBI's GTFS data management platform.

License: MIT License

Java 99.21% HTML 0.05% Shell 0.31% JavaScript 0.17% Python 0.22% Dockerfile 0.04%

datatools-server's People

Contributors

Stargazers

Watchers

Forkers

catalogueglobal l-vincent-l josetiagobispo okilian a1ex4nd3r causecode vijistepsup apinf robbi5 metrotranscom artemvn pibe86 hjtamaral mjydpro knaaptime aniljambukiya digital-botschafter-und-mehr wolli openmaas colombiatransit wkulesza bakhshi realcity vesavlad aakoo bijanmmarkes mvanlaar wtimmerman-fitp bulksecuritygeneratorprojectv2 mfdz rj-smtr logos-code goeuropa oblivion-studios saiful7 nautilustechnologyau cpresori donbot2000 et430-rosenheim vuski

datatools-server's Issues

Not able to upload shapefile to the feed

When we try to upload shapefile to the file nothing happen, file not upload neither there is validation error, the upload popup remain in place

Preflight request with the OPTIONS are needed

Issue by zbouziane
Tuesday Mar 07, 2017 at 11:49 GMT
Originally opened as catalogueglobal#2

Especially in FeedVersionController::register, FeedSourceController::register, ProjectController::register and UserController::register.

ArrayIndexOutOfBoundsException thrown during a GTFS file upload

Issue by zbouziane
Tuesday Sep 19, 2017 at 08:08 GMT
Originally opened as catalogueglobal#36

On mongo-pojo branch at commit number 2b91f71e98384f51e0d57d973a00e7c5731ab40a:

sept. 19 09:56:55.178 [pool-2-thread-1] INFO  com.conveyal.osmlib.PBFInput - Done parsing PBF.
sept. 19 09:56:55.179 [pool-2-thread-1] INFO  com.conveyal.osmlib.PBFInput - Read 0 nodes, 0 ways, 0 relations.
sept. 19 09:56:55.282 [pool-2-thread-1] INFO  com.conveyal.r5.streets.StreetLayer - Making street edges from OSM ways...
sept. 19 09:56:55.293 [pool-2-thread-1] WARN  c.c.d.manager.models.FeedVersion - Unknown error encountered while building network for TISSEO2-20170919T095640-02-2341da9d-c09c-48fc-8dbd-7e94c7683f0f.zip.
java.lang.ArrayIndexOutOfBoundsException: Attempt to seek beyond end of edge store
	at com.conveyal.r5.streets.EdgeStore$Edge.seek(EdgeStore.java:348)
	at com.conveyal.r5.streets.StreetLayer.loadFromOsm(StreetLayer.java:273)
	at com.conveyal.r5.streets.StreetLayer.loadFromOsm(StreetLayer.java:170)
	at com.conveyal.r5.transit.TransportNetwork.fromFiles(TransportNetwork.java:166)
	at com.conveyal.r5.transit.TransportNetwork.fromFeeds(TransportNetwork.java:133)
	at com.conveyal.datatools.manager.models.FeedVersion.buildTransportNetwork(FeedVersion.java:522)
	at com.conveyal.datatools.manager.jobs.BuildTransportNetworkJob.run(BuildTransportNetworkJob.java:31)
	at java.util.ArrayList.forEach(ArrayList.java:1249)
	at com.conveyal.datatools.common.status.MonitorableJob.jobFinished(MonitorableJob.java:83)
	at com.conveyal.datatools.manager.jobs.ValidateFeedJob.run(ValidateFeedJob.java:50)
	at com.conveyal.datatools.manager.jobs.ProcessSingleFeedJob.run(ProcessSingleFeedJob.java:55)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
`
``

How do I configure the backend to point to the frontend anywhere but Amazon?

Issue by loque
Friday Aug 18, 2017 at 17:10 GMT
Originally opened as catalogueglobal#31

Am I required to serve the frontend from Amazon's S3 Bucket?
I know I can avoid using S3 for the gtfs cache configuring application.data.use_s3_storage to false at datatools-server:server.yml. But when it comes to the assets I only found application.assets_bucket and no way to configure a local path instead.

Bugfixes downloads

Issue by l-vincent-l
Thursday Aug 03, 2017 at 09:43 GMT
Originally opened as catalogueglobal#27

We were not allowed to download a file when the user is not logged in.

l-vincent-l included the following code: https://github.com/catalogueglobal/datatools-server/pull/27/commits

fix deployment of bundle to s3 and update aws sdk

Issue by demory
Wednesday Aug 09, 2017 at 00:43 GMT
Originally opened as catalogueglobal#29

demory included the following code: https://github.com/catalogueglobal/datatools-server/pull/29/commits

Fix transfer managers to not shut down s3 client

Issue by evansiroky
Wednesday Aug 09, 2017 at 01:15 GMT
Originally opened as catalogueglobal#30

evansiroky included the following code: https://github.com/catalogueglobal/datatools-server/pull/30/commits

Reversed Trip Validation raises a NullPointerException

Issue by zbouziane
Friday Apr 28, 2017 at 13:25 GMT
Originally opened as catalogueglobal#15

...
13:21:29.491 [Thread-11] INFO  com.conveyal.gtfs.GTFSFeed - DuplicateStopsValidator finished in 2092 milliseconds.
13:21:29.492 [Thread-11] INFO  com.conveyal.gtfs.GTFSFeed - HopSpeedsReasonableValidator finished in 0 milliseconds.
13:21:29.630 [Thread-11] INFO  com.conveyal.gtfs.GTFSFeed - MisplacedStopValidator finished in 137 milliseconds.
13:21:29.632 [Thread-11] INFO  com.conveyal.gtfs.GTFSFeed - NamesValidator finished in 1 milliseconds.
13:21:32.257 [Thread-11] INFO  com.conveyal.gtfs.GTFSFeed - OverlappingTripsValidator finished in 2625 milliseconds.
13:21:33.109 [Thread-11] ERROR com.conveyal.gtfs.GTFSFeed - Could not run ReversedTripsValidator validator.
java.lang.NullPointerException
	at com.conveyal.gtfs.validator.ReversedTripsValidator.validate(ReversedTripsValidator.java:68)
	at com.conveyal.gtfs.GTFSFeed.validate(GTFSFeed.java:284)
	at com.conveyal.gtfs.GTFSFeed.validate(GTFSFeed.java:301)
	at com.conveyal.datatools.manager.models.FeedVersion.validate(FeedVersion.java:282)
	at com.conveyal.datatools.manager.jobs.ValidateFeedJob.run(ValidateFeedJob.java:45)
	at java.lang.Thread.run(Thread.java:745)
13:21:35.756 [Thread-11] INFO  com.conveyal.gtfs.GTFSFeed - TripTimesValidator finished in 2643 milliseconds.
13:21:35.761 [Thread-11] INFO  com.conveyal.gtfs.GTFSFeed - UnusedStopValidator finished in 4 milliseconds.
13:21:35.761 [Thread-11] INFO  com.conveyal.gtfs.GTFSFeed - 8 validators completed in 8362 milliseconds.
13:21:35.761 [Thread-11] INFO  c.c.d.manager.models.FeedVersion - Calculating stats...
13:21:37.948 [Thread-11] INFO  c.c.d.manager.models.FeedVersion - Calculating # of trips per date of service
13:21:51.062 [Thread-11] INFO  c.c.d.manager.models.FeedVersion - Total errors after validation: 9
13:21:51.063 [Thread-11] INFO  c.c.d.manager.models.FeedVersion - Uploading validation json to S3
...

Add datagouvfr extension

Issue by l-vincent-l
Thursday Aug 03, 2017 at 10:24 GMT
Originally opened as catalogueglobal#28

We were not allowed to download a file when the user is not logged in.

l-vincent-l included the following code: https://github.com/catalogueglobal/datatools-server/pull/28/commits

Algarve data import/conversion to GTFS

Issue by landonreed
Wednesday Sep 20, 2017 at 16:46 GMT
Originally opened as catalogueglobal#39

This is the initial work to import Algarve transit data from a SQL Server export file (.bak) and pull the data into the data manager for conversion to GTFS.

landonreed included the following code: https://github.com/catalogueglobal/datatools-server/pull/39/commits

Create version from snapshot failing for SamTrans

        at org.mapdb.SerializerPojo.deserializeUnknownHeader(SerializerPojo.java:483)
        at org.mapdb.SerializerBase.deserialize3(SerializerBase.java:1216)
        at org.mapdb.SerializerBase.deserialize(SerializerBase.java:1132)
        at org.mapdb.SerializerBase.deserialize(SerializerBase.java:867)
        at org.mapdb.SerializerPojo.deserialize(SerializerPojo.java:701)
        at org.mapdb.BTreeKeySerializer$BasicKeySerializer.deserialize(BTreeKeySerializer.java:89)
        at org.mapdb.BTreeMap$NodeSerializer.deserialize(BTreeMap.java:440)
        at org.mapdb.BTreeMap$NodeSerializer.deserialize(BTreeMap.java:288)
        at org.mapdb.Store.deserialize(Store.java:297)
        at org.mapdb.StoreDirect.get2(StoreDirect.java:486)
        at org.mapdb.StoreWAL.get2(StoreWAL.java:336)
        at org.mapdb.StoreWAL.get(StoreWAL.java:320)
        at org.mapdb.Caches$HashTable.get(Caches.java:246)
        at org.mapdb.EngineWrapper.get(EngineWrapper.java:58)
        at org.mapdb.TxEngine.get(TxEngine.java:192)
        at org.mapdb.TxEngine$Tx.getNoLock(TxEngine.java:427)
        at org.mapdb.TxEngine$Tx.get(TxEngine.java:402)
        at org.mapdb.EngineWrapper.get(EngineWrapper.java:58)
        at org.mapdb.TxEngine.get(TxEngine.java:192)
        at org.mapdb.BTreeMap.<init>(BTreeMap.java:541)
        at org.mapdb.DB.getTreeMap(DB.java:805)
        at org.mapdb.DB$BTreeMapMaker.makeOrGet(DB.java:667)
        at com.conveyal.datatools.editor.datastore.DatabaseTx.getMap(DatabaseTx.java:35)
        at com.conveyal.datatools.editor.datastore.FeedTx.buildSecondaryIndices(FeedTx.java:171)
        at com.conveyal.datatools.editor.datastore.FeedTx.<init>(FeedTx.java:112)
        at com.conveyal.datatools.editor.datastore.FeedTx.<init>(FeedTx.java:94)
        at com.conveyal.datatools.editor.datastore.VersionedDataStore.getFeedTx(VersionedDataStore.java:57)
        at com.conveyal.datatools.editor.jobs.ProcessGtfsSnapshotExport.run(ProcessGtfsSnapshotExport.java:81)
        at com.conveyal.datatools.editor.controllers.api.SnapshotController.writeSnapshotAsGtfs(SnapshotController.java:255)
        at com.conveyal.datatools.editor.controllers.api.SnapshotController.writeSnapshotAsGtfs(SnapshotController.java:269)
        at com.conveyal.datatools.manager.jobs.CreateFeedVersionFromSnapshotJob.run(CreateFeedVersionFromSnapshotJob.java:41)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
        at java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:387)
        at java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:396)
        at org.mapdb.SerializerPojo.deserializeUnknownHeader(SerializerPojo.java:450)
        ... 31 more
22:26:38.299 [Thread-11] ERROR c.c.d.m.j.CreateFeedVersionFromSnapshotJob - Unable to create temp file for snapshot```

Update VEX URL

Issue by abyrd
Wednesday Jun 28, 2017 at 04:13 GMT
Originally opened as catalogueglobal#20

Data tools is the only Conveyal product still using the old C VEX URL pattern. We should update it so our VEX nginx proxy doesn't need to contain URL rewrite rules. Alternatively update everything to use the old VEX URL pattern which @mattwigway pointed out is probably better since it uses named URL parameters.

Fix Vex Url to conform to osm-lib url pattern

Issue by aguiraf
Friday Jul 28, 2017 at 17:05 GMT
Originally opened as catalogueglobal#25

Fix #20

aguiraf included the following code: https://github.com/catalogueglobal/datatools-server/pull/25/commits

Run server with no auth for local development

Issue by landonreed
Monday Jun 12, 2017 at 20:53 GMT
Originally opened as catalogueglobal#18

Remove server.yml from git

Issue by l-vincent-l
Thursday Aug 03, 2017 at 09:39 GMT
Originally opened as catalogueglobal#26

This configuration file shouldn't be gitted, we provide an example file to the user instead so he can copy & paste it.

l-vincent-l included the following code: https://github.com/catalogueglobal/datatools-server/pull/26/commits

Feeds are uploaded to locale FS but downloaded from s3 bucket ?

Issue by zbouziane
Tuesday Apr 18, 2017 at 13:03 GMT
Originally opened as catalogueglobal#8

14:58:30.111 [main] INFO  c.c.datatools.manager.DataManager - Loading env.yml: configurations/default/env.yml
14:58:30.115 [main] INFO  c.c.datatools.manager.DataManager - Loading server.yml: configurations/default/server.yml
14:58:30.191 [main] INFO  c.c.d.manager.persistence.DataStore - /Users/zbouziane/Data/cataloguedev/mapdb/projects
14:58:30.375 [main] INFO  c.c.datatools.manager.DataManager - Initializing gtfs-api for bucket cataloguedev/gtfs/ and cache dir /Users/zbouziane/Data/cataloguedev/gtfs
log4j:WARN No appenders could be found for logger (com.amazonaws.AmazonWebServiceClient).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
14:58:31.078 [main] INFO  com.conveyal.gtfs.GTFSCache - Using bucket cataloguedev for GTFS Cache
14:58:31.109 [Thread-1] INFO  org.eclipse.jetty.util.log - Logging initialized @6536ms
14:58:31.178 [Thread-1] INFO  s.e.jetty.EmbeddedJettyServer - == Spark has ignited ...
14:58:31.179 [Thread-1] INFO  s.e.jetty.EmbeddedJettyServer - >> Listening on 0.0.0.0:4000
14:58:31.182 [Thread-1] INFO  org.eclipse.jetty.server.Server - jetty-9.3.z-SNAPSHOT
14:58:31.313 [Thread-1] INFO  o.e.jetty.server.ServerConnector - Started ServerConnector@d6e9a92{HTTP/1.1,[http/1.1]}{0.0.0.0:4000}
14:58:31.316 [Thread-1] INFO  org.eclipse.jetty.server.Server - Started @6744ms
14:58:31.388 [main] WARN  c.c.d.m.c.api.GtfsApiController - No extension provided for GTFS API
14:58:31.390 [pool-1-thread-1] INFO  c.c.d.manager.jobs.FeedUpdater - Fetching feeds...
14:58:31.390 [pool-1-thread-1] INFO  c.c.d.manager.jobs.FeedUpdater - Current eTag list {}
14:58:31.405 [main] WARN  c.c.datatools.manager.DataManager - Config property modules.gtfsplus.enabled not found
14:58:31.405 [main] WARN  c.c.datatools.manager.DataManager - Config property modules.gtfsplus.enabled not found
14:58:31.405 [main] WARN  c.c.datatools.manager.DataManager - Config property modules.gtfsplus.enabled not found
14:58:31.413 [main] WARN  c.c.datatools.manager.DataManager - Config property modules.dump.enabled not found
14:58:31.413 [main] WARN  c.c.datatools.manager.DataManager - Config property modules.dump.enabled not found
14:58:31.413 [main] WARN  c.c.datatools.manager.DataManager - Config property modules.dump.enabled not found
14:58:31.417 [main] WARN  c.c.datatools.manager.DataManager - Config property extensions.mtc.enabled not found
14:58:31.418 [main] WARN  c.c.datatools.manager.DataManager - Config property extensions.mtc.enabled not found
14:58:31.418 [main] WARN  c.c.datatools.manager.DataManager - Config property extensions.mtc.enabled not found
14:58:31.418 [main] INFO  c.c.datatools.manager.DataManager - Registering TransitLand Resource
14:58:31.419 [main] INFO  c.c.datatools.manager.DataManager - Registering TransitFeeds Resource
14:58:31.974 [pool-1-thread-1] INFO  c.c.d.manager.jobs.FeedUpdater - No feeds updated...
14:58:42.352 [qtp891452239-15] INFO  c.c.d.m.c.api.ProjectController - found projects: 0
/api/manager/public/region
14:58:42.439 [qtp891452239-17] INFO  c.c.d.manager.persistence.DataStore - /Users/zbouziane/Data/cataloguedev/mapdb/region
14:58:45.460 [qtp891452239-20] INFO  c.c.d.m.c.api.ProjectController - found projects: 0
14:58:51.244 [qtp891452239-18] INFO  c.c.d.m.c.api.ProjectController - found projects: 0
14:59:00.327 [qtp891452239-21] INFO  c.c.d.manager.persistence.DataStore - /Users/zbouziane/Data/cataloguedev/mapdb/feedsources
14:59:00.429 [qtp891452239-20] INFO  c.c.d.m.c.api.ProjectController - found projects: 1
developer.digitalfactory@gmail.com
setting fs project
14:59:17.457 [qtp891452239-17] INFO  c.c.d.manager.persistence.DataStore - /Users/zbouziane/Data/cataloguedev/mapdb/feedversions
14:59:17.500 [qtp891452239-17] INFO  c.c.d.manager.persistence.DataStore - /Users/zbouziane/Data/cataloguedev/mapdb/externalFeedSourceProperties
developer.digitalfactory@gmail.com
14:59:44.113 [qtp891452239-20] INFO  c.c.d.m.c.api.FeedVersionController - Saving feed from upload <FeedSource tampere (10e9ec1c-a6c7-418e-95f9-9eb4d911f167)>
14:59:44.113 [qtp891452239-20] INFO  c.c.d.manager.persistence.FeedStore - Writing file to /Users/zbouziane/Data/cataloguedev/gtfs/tampere-20170418T145943+02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167.zip
14:59:44.182 [qtp891452239-20] INFO  c.c.d.manager.persistence.FeedStore - Copying version to latest <FeedSource tampere (10e9ec1c-a6c7-418e-95f9-9eb4d911f167)>
14:59:44.200 [qtp891452239-20] INFO  c.c.d.manager.models.FeedVersion - New GTFS file saved: tampere-20170418T145943+02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167.zip
14:59:44.209 [qtp891452239-20] INFO  c.c.d.m.c.api.FeedVersionController - Last modified: Fri Dec 02 09:44:10 CET 2016
14:59:44.214 [qtp891452239-20] INFO  c.c.d.manager.persistence.FeedStore - Downloading feed from s3
14:59:44.366 [qtp891452239-20] ERROR c.c.d.manager.persistence.FeedStore - Error downloading from s3
com.amazonaws.services.s3.model.AmazonS3Exception: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: 25A928491624EB60), S3 Extended Request ID: DM7fWaJ+9bKZcoXEBrjDDBS8nwwgxQwoCek7HdEtky+y1QQxr0VnfF/B1HnZxz48gl6fPkY1h60=
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1579)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1249)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4191)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4138)
	at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1385)
	at com.conveyal.datatools.manager.persistence.FeedStore.getFeed(FeedStore.java:159)
	at com.conveyal.datatools.manager.models.FeedVersion.getGtfsFile(FeedVersion.java:155)
	at com.conveyal.datatools.manager.models.FeedVersion.hash(FeedVersion.java:441)
	at com.conveyal.datatools.manager.controllers.api.FeedVersionController.createFeedVersion(FeedVersionController.java:134)
	at spark.ResponseTransformerRouteImpl$1.handle(ResponseTransformerRouteImpl.java:47)
	at spark.http.matching.Routes.execute(Routes.java:61)
	at spark.http.matching.MatcherFilter.doFilter(MatcherFilter.java:126)
	at spark.embeddedserver.jetty.JettyHandler.doHandle(JettyHandler.java:50)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:189)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
	at org.eclipse.jetty.server.Server.handle(Server.java:517)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
	at java.lang.Thread.run(Thread.java:745)
14:59:44.989 [qtp891452239-20] INFO  c.c.d.e.j.ProcessGtfsSnapshotMerge - GTFS Snapshot Merge for feedVersion tampere-20170418T145943+02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167.zip
14:59:44.991 [Thread-11] INFO  c.c.d.manager.jobs.ValidateFeedJob - Running ValidateFeedJob for tampere-20170418T145943+02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167.zip
14:59:44.993 [Thread-11] INFO  c.c.d.manager.auth.Auth0Users - Auth0 getUsers URL=https://cataloguetest.eu.auth0.com/api/v2/users/auth0%7C57a1f0c5b8155e90194e5087
14:59:45.233 [Thread-11] INFO  com.conveyal.gtfs.GTFSCache - Attempting to download cached GTFS MapDB.
14:59:45.368 [Thread-11] WARN  com.conveyal.gtfs.GTFSCache - DB file for key gtfs/tampere-20170418T145943-02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167 does not exist on S3.
14:59:45.368 [Thread-11] INFO  com.conveyal.gtfs.GTFSCache - Loading feed from local cache directory...
14:59:45.368 [Thread-11] INFO  com.conveyal.gtfs.GTFSCache - Feed not found locally, downloading from S3.
14:59:45.399 [Thread-11] ERROR com.conveyal.gtfs.GTFSCache - Could not download feed at s3://cataloguedev/gtfs/tampere-20170418T145943-02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167.
14:59:45.400 [Thread-11] WARN  c.c.d.manager.models.FeedVersion - No GTFS feed exists for version: tampere-20170418T145943+02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167.zip
Building network
14:59:45.443 [Thread-12] INFO  c.c.d.e.j.ProcessGtfsSnapshotMerge - Cleared old data
14:59:45.444 [Thread-12] INFO  com.conveyal.gtfs.GTFSCache - Attempting to download cached GTFS MapDB.
14:59:45.478 [Thread-12] WARN  com.conveyal.gtfs.GTFSCache - DB file for key gtfs/tampere-20170418T145943-02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167 does not exist on S3.
14:59:45.478 [Thread-12] INFO  com.conveyal.gtfs.GTFSCache - Loading feed from local cache directory...
14:59:45.478 [Thread-12] INFO  com.conveyal.gtfs.GTFSCache - Feed not found locally, downloading from S3.
14:59:45.506 [Thread-12] ERROR com.conveyal.gtfs.GTFSCache - Could not download feed at s3://cataloguedev/gtfs/tampere-20170418T145943-02-10e9ec1c-a6c7-418e-95f9-9eb4d911f167.
java.lang.NullPointerException
	at com.conveyal.datatools.manager.models.FeedVersion.getGtfsFeed(FeedVersion.java:181)
	at com.conveyal.datatools.editor.jobs.ProcessGtfsSnapshotMerge.run(ProcessGtfsSnapshotMerge.java:120)
	at java.lang.Thread.run(Thread.java:745)
Exception in thread "Thread-12" spark.HaltException
	at spark.Service.halt(Service.java:437)
	at spark.Spark.halt(Spark.java:854)
	at com.conveyal.datatools.editor.jobs.ProcessGtfsSnapshotMerge.run(ProcessGtfsSnapshotMerge.java:459)
	at java.lang.Thread.run(Thread.java:745)

Add devops features to the project

Issue by cgalant
Friday Jul 07, 2017 at 13:36 GMT
Originally opened as catalogueglobal#21

fix tests issue
improve logging
build docker image
refactoring the pom.xml for CI integration

cgalant included the following code: https://github.com/catalogueglobal/datatools-server/pull/21/commits

New Test Failure

Issue by zbouziane
Wednesday Apr 19, 2017 at 15:19 GMT
Originally opened as catalogueglobal#10

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.conveyal.datatools.editor.ProcessGtfsSnapshotMergeTest
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/zbouziane/.m2/repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/zbouziane/.m2/repository/org/slf4j/slf4j-simple/1.7.10/slf4j-simple-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
17:12:57,786 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
17:12:57,787 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
17:12:57,787 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [file:/Users/zbouziane/tmp/tmp/datatools-server/target/classes/logback.xml]
17:12:57,788 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs multiple times on the classpath.
17:12:57,788 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/Users/zbouziane/.m2/repository/com/conveyal/osm-lib/1.1.0-SNAPSHOT/osm-lib-1.1.0-SNAPSHOT.jar!/logback.xml]
17:12:57,788 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [file:/Users/zbouziane/tmp/tmp/datatools-server/target/classes/logback.xml]
17:12:57,973 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
17:13:02,984 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
17:13:02,991 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
17:13:03,017 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
17:13:03,069 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to DEBUG
17:13:03,069 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
17:13:03,070 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
17:13:03,071 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@22eeefeb - Registering current configuration as safe fallback point

SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
17/04/19 17:13:03.080 [main] INFO  com.conveyal.datatools.DatatoolsTest - DatatoolsTest setup
17/04/19 17:13:03.272 [main] INFO  c.c.datatools.manager.DataManager - Loading env.yml: configurations/default/env.yml.tmp
17/04/19 17:13:03.274 [main] INFO  c.c.datatools.manager.DataManager - Loading server.yml: configurations/default/server.yml
17/04/19 17:13:03.400 [main] INFO  c.c.d.manager.persistence.DataStore - /private/tmp/projects
17/04/19 17:13:03.574 [main] WARN  c.c.datatools.manager.DataManager - Config property application.data.gtfs_s3_bucket not found
17/04/19 17:13:03.575 [main] WARN  c.c.datatools.manager.DataManager - Config property application.data.gtfs_s3_bucket not found
17/04/19 17:13:03.584 [main] WARN  c.c.datatools.manager.DataManager - Config property application.data.s3_credentials_file not found
17/04/19 17:13:03.584 [main] WARN  c.c.datatools.manager.DataManager - Config property application.data.s3_credentials_file not found
17/04/19 17:13:03.584 [main] INFO  c.c.datatools.manager.DataManager - Initializing gtfs-api cache locally (no s3 bucket) /tmp
17/04/19 17:13:04.450 [main] INFO  com.conveyal.gtfs.BaseGTFSCache - No bucket specified; GTFS Cache will run locally
17/04/19 17:13:04.489 [main] DEBUG spark.route.Routes - Adds route: after, +/*paths, spark.FilterImpl$1@30af5b6b
17/04/19 17:13:04.522 [Thread-1] DEBUG org.eclipse.jetty.util.log - Logging to Logger[org.eclipse.jetty.util.log] via org.eclipse.jetty.util.log.Slf4jLog
17/04/19 17:13:04.528 [Thread-1] INFO  org.eclipse.jetty.util.log - Logging initialized @7553ms
17/04/19 17:13:04.539 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - spark.embeddedserver.jetty.JettyHandler@740097e2 added {org.eclipse.jetty.server.session.HashSessionManager@9af482e,AUTO}
17/04/19 17:13:04.559 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/project/:id, spark.ResponseTransformerRouteImpl$1@2c1156a7
17/04/19 17:13:04.559 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - org.eclipse.jetty.server.Server@2e01be57 added {qtp1978280845{STOPPED,8<=0<=200,i=0,q=0},AUTO}
17/04/19 17:13:04.561 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/project, spark.ResponseTransformerRouteImpl$1@4f1bfe23
17/04/19 17:13:04.563 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/project, spark.ResponseTransformerRouteImpl$1@71529963
17/04/19 17:13:04.568 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/project/:id, spark.ResponseTransformerRouteImpl$1@5adb0db3
17/04/19 17:13:04.625 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/project/:id, spark.ResponseTransformerRouteImpl$1@7241a47d
17/04/19 17:13:04.626 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/project/:id/thirdPartySync/:type, spark.ResponseTransformerRouteImpl$1@5dcd8c7a
17/04/19 17:13:04.628 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/project/:id/fetch, spark.ResponseTransformerRouteImpl$1@253d9f73
17/04/19 17:13:04.630 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/project/:id/deployPublic, spark.ResponseTransformerRouteImpl$1@8dbfffb
17/04/19 17:13:04.631 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/project/:id/download, spark.RouteImpl$1@a22cb6a
17/04/19 17:13:04.632 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/project/:id, spark.ResponseTransformerRouteImpl$1@2e2ff723
17/04/19 17:13:04.634 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/project, spark.ResponseTransformerRouteImpl$1@2445445a
17/04/19 17:13:04.644 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedsource/:id, spark.ResponseTransformerRouteImpl$1@63798ca7
17/04/19 17:13:04.645 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedsource, spark.ResponseTransformerRouteImpl$1@5674e1f2
17/04/19 17:13:04.646 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/feedsource, spark.ResponseTransformerRouteImpl$1@32f232a5
17/04/19 17:13:04.648 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - HttpConnectionFactory@1f00700c[HTTP/1.1] added {HttpConfiguration@5520a96e{32768/8192,8192/8192,https://:0,[]},POJO}
17/04/19 17:13:04.649 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/feedsource/:id, spark.ResponseTransformerRouteImpl$1@e8df99a
17/04/19 17:13:04.650 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/feedsource/:id/updateExternal, spark.ResponseTransformerRouteImpl$1@517566b
17/04/19 17:13:04.651 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - ServerConnector@1a06506c{null,[]}{0.0.0.0:0} added {org.eclipse.jetty.server.Server@2e01be57,UNMANAGED}
17/04/19 17:13:04.652 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/feedsource/:id, spark.ResponseTransformerRouteImpl$1@2df6226d
17/04/19 17:13:04.652 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - ServerConnector@1a06506c{null,[]}{0.0.0.0:0} added {qtp1978280845{STOPPED,8<=0<=200,i=0,q=0},AUTO}
17/04/19 17:13:04.653 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - ServerConnector@1a06506c{null,[]}{0.0.0.0:0} added {org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@112a1b5d,AUTO}
17/04/19 17:13:04.653 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - ServerConnector@1a06506c{null,[]}{0.0.0.0:0} added {org.eclipse.jetty.io.ArrayByteBufferPool@2a5887b5,POJO}
17/04/19 17:13:04.654 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - ServerConnector@1a06506c{null,[http/1.1]}{0.0.0.0:0} added {HttpConnectionFactory@1f00700c[HTTP/1.1],AUTO}
17/04/19 17:13:04.654 [Thread-1] DEBUG o.e.jetty.server.ServerConnector - ServerConnector@1a06506c{HTTP/1.1,[http/1.1]}{0.0.0.0:0} added HttpConnectionFactory@1f00700c[HTTP/1.1]
17/04/19 17:13:04.657 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - ServerConnector@1a06506c{HTTP/1.1,[http/1.1]}{0.0.0.0:0} added {org.eclipse.jetty.server.ServerConnector$ServerConnectorManager@31efc5b5,MANAGED}
17/04/19 17:13:04.658 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - org.eclipse.jetty.server.Server@2e01be57 added {ServerConnector@1a06506c{HTTP/1.1,[http/1.1]}{0.0.0.0:4000},AUTO}
17/04/19 17:13:04.662 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - org.eclipse.jetty.server.Server@2e01be57 added {spark.embeddedserver.jetty.JettyHandler@740097e2,MANAGED}
17/04/19 17:13:04.662 [Thread-1] INFO  s.e.jetty.EmbeddedJettyServer - == Spark has ignited ...
17/04/19 17:13:04.662 [Thread-1] INFO  s.e.jetty.EmbeddedJettyServer - >> Listening on 0.0.0.0:4000
17/04/19 17:13:04.663 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting org.eclipse.jetty.server.Server@2e01be57
17/04/19 17:13:04.667 [Thread-1] INFO  org.eclipse.jetty.server.Server - jetty-9.3.6.v20151106
17/04/19 17:13:04.701 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/feedsource/:id/fetch, spark.ResponseTransformerRouteImpl$1@55322aab
17/04/19 17:13:04.703 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/feedsource/:id, spark.ResponseTransformerRouteImpl$1@50468873
17/04/19 17:13:04.705 [Thread-1] DEBUG o.e.j.server.handler.AbstractHandler - starting org.eclipse.jetty.server.Server@2e01be57
17/04/19 17:13:04.705 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/feedsource, spark.ResponseTransformerRouteImpl$1@76884e4b
17/04/19 17:13:04.705 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting qtp1978280845{STOPPED,8<=0<=200,i=0,q=0}
17/04/19 17:13:04.707 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7732ms qtp1978280845{STARTED,8<=8<=200,i=7,q=0}
17/04/19 17:13:04.707 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting spark.embeddedserver.jetty.JettyHandler@740097e2
17/04/19 17:13:04.707 [Thread-1] DEBUG o.e.j.server.handler.AbstractHandler - starting spark.embeddedserver.jetty.JettyHandler@740097e2
17/04/19 17:13:04.707 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting org.eclipse.jetty.server.session.HashSessionManager@9af482e
17/04/19 17:13:04.712 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - org.eclipse.jetty.server.session.HashSessionManager@9af482e added {org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@1c1018ce,MANAGED}
17/04/19 17:13:04.715 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting org.eclipse.jetty.server.session.HashSessionIdManager@1fa25d56
17/04/19 17:13:04.715 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7741ms org.eclipse.jetty.server.session.HashSessionIdManager@1fa25d56
17/04/19 17:13:04.716 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - org.eclipse.jetty.server.Server@2e01be57 added {org.eclipse.jetty.server.session.HashSessionIdManager@1fa25d56,MANAGED}
17/04/19 17:13:04.716 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - org.eclipse.jetty.server.session.HashSessionManager@9af482e added {org.eclipse.jetty.server.session.HashSessionIdManager@1fa25d56,UNMANAGED}
17/04/19 17:13:04.716 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@1c1018ce
17/04/19 17:13:04.716 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedversion/:id, spark.ResponseTransformerRouteImpl$1@62727399
17/04/19 17:13:04.717 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7742ms org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@1c1018ce
17/04/19 17:13:04.717 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedversion/:id/download, spark.RouteImpl$1@130dca52
17/04/19 17:13:04.720 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7745ms org.eclipse.jetty.server.session.HashSessionManager@9af482e
17/04/19 17:13:04.720 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7746ms spark.embeddedserver.jetty.JettyHandler@740097e2
17/04/19 17:13:04.721 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting ServerConnector@1a06506c{HTTP/1.1,[http/1.1]}{0.0.0.0:4000}
17/04/19 17:13:04.728 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedversion/:id/downloadtoken, spark.ResponseTransformerRouteImpl$1@15dcfae7
17/04/19 17:13:04.732 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedversion/:id/validation, spark.ResponseTransformerRouteImpl$1@115667d
17/04/19 17:13:04.734 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/feedversion/:id/validate, spark.ResponseTransformerRouteImpl$1@497486b3
17/04/19 17:13:04.744 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedversion/:id/isochrones, spark.ResponseTransformerRouteImpl$1@6127a7e
17/04/19 17:13:04.746 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedversion, spark.ResponseTransformerRouteImpl$1@76012793
17/04/19 17:13:04.748 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/feedversion, spark.ResponseTransformerRouteImpl$1@4dd02341
17/04/19 17:13:04.749 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/feedversion/fromsnapshot, spark.ResponseTransformerRouteImpl$1@495b0487
17/04/19 17:13:04.750 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - ServerConnector@1a06506c{HTTP/1.1,[http/1.1]}{0.0.0.0:4000} added {sun.nio.ch.ServerSocketChannelImpl[/0:0:0:0:0:0:0:0:4000],POJO}
17/04/19 17:13:04.751 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@112a1b5d
17/04/19 17:13:04.751 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7777ms org.eclipse.jetty.util.thread.ScheduledExecutorScheduler@112a1b5d
17/04/19 17:13:04.752 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting HttpConnectionFactory@1f00700c[HTTP/1.1]
17/04/19 17:13:04.753 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7777ms HttpConnectionFactory@1f00700c[HTTP/1.1]
17/04/19 17:13:04.753 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting org.eclipse.jetty.server.ServerConnector$ServerConnectorManager@31efc5b5
17/04/19 17:13:04.762 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/feedversion/:id/rename, spark.ResponseTransformerRouteImpl$1@93cf163
17/04/19 17:13:04.764 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/feedversion/:id/publish, spark.ResponseTransformerRouteImpl$1@25bfcafd
17/04/19 17:13:04.767 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/feedversion/:id, spark.ResponseTransformerRouteImpl$1@2a76840c
17/04/19 17:13:04.768 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/feedversion, spark.ResponseTransformerRouteImpl$1@7cf6a5f9
17/04/19 17:13:04.769 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting org.eclipse.jetty.io.ManagedSelector@5c0b94f8 id=0 keys=-1 selected=-1
17/04/19 17:13:04.769 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/feedversion/:id/validation, spark.ResponseTransformerRouteImpl$1@66565121
17/04/19 17:13:04.770 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/feedversion/:id/downloadtoken, spark.ResponseTransformerRouteImpl$1@6179e425
17/04/19 17:13:04.771 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/downloadfeed/:token, spark.RouteImpl$1@2d6764b2
17/04/19 17:13:04.776 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/region/:id, spark.ResponseTransformerRouteImpl$1@6399551e
17/04/19 17:13:04.779 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/region, spark.ResponseTransformerRouteImpl$1@5d5f10b2
17/04/19 17:13:04.781 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/region, spark.ResponseTransformerRouteImpl$1@53bd8fca
17/04/19 17:13:04.783 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7806ms org.eclipse.jetty.io.ManagedSelector@5c0b94f8 id=0 keys=0 selected=0
17/04/19 17:13:04.783 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/region/:id, spark.ResponseTransformerRouteImpl$1@5cde6747
17/04/19 17:13:04.784 [Thread-1] DEBUG o.e.j.util.thread.QueuedThreadPool - queue org.eclipse.jetty.io.ManagedSelector@5c0b94f8 id=0 keys=0 selected=0
17/04/19 17:13:04.786 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/region/:id, spark.ResponseTransformerRouteImpl$1@3ed242a4
17/04/19 17:13:04.787 [qtp1978280845-14] DEBUG o.e.j.util.thread.QueuedThreadPool - run org.eclipse.jetty.io.ManagedSelector@5c0b94f8 id=0 keys=0 selected=0
17/04/19 17:13:04.787 [qtp1978280845-14] DEBUG o.e.j.u.t.s.ExecuteProduceConsume - EPR Idle/org.eclipse.jetty.io.ManagedSelector$SelectorProducer@162fba2a execute
17/04/19 17:13:04.788 [qtp1978280845-14] DEBUG o.e.j.u.t.s.ExecuteProduceConsume - EPR Prod/org.eclipse.jetty.io.ManagedSelector$SelectorProducer@162fba2a produce enter
17/04/19 17:13:04.788 [qtp1978280845-14] DEBUG o.e.j.u.t.s.ExecuteProduceConsume - EPR Prod/org.eclipse.jetty.io.ManagedSelector$SelectorProducer@162fba2a producing
17/04/19 17:13:04.788 [qtp1978280845-14] DEBUG org.eclipse.jetty.io.ManagedSelector - Selector loop waiting on select
17/04/19 17:13:04.791 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/region/:id, spark.ResponseTransformerRouteImpl$1@1fdf1c5
17/04/19 17:13:04.793 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - starting org.eclipse.jetty.io.ManagedSelector@31b31a76 id=1 keys=-1 selected=-1
17/04/19 17:13:04.793 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/region, spark.ResponseTransformerRouteImpl$1@7d64e326
17/04/19 17:13:04.794 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7819ms org.eclipse.jetty.io.ManagedSelector@31b31a76 id=1 keys=0 selected=0
17/04/19 17:13:04.795 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/public/seedregions, spark.ResponseTransformerRouteImpl$1@68be8808
17/04/19 17:13:04.796 [Thread-1] DEBUG o.e.j.util.thread.QueuedThreadPool - queue org.eclipse.jetty.io.ManagedSelector@31b31a76 id=1 keys=0 selected=0
17/04/19 17:13:04.799 [qtp1978280845-15] DEBUG o.e.j.util.thread.QueuedThreadPool - run org.eclipse.jetty.io.ManagedSelector@31b31a76 id=1 keys=0 selected=0
17/04/19 17:13:04.799 [qtp1978280845-15] DEBUG o.e.j.u.t.s.ExecuteProduceConsume - EPR Idle/org.eclipse.jetty.io.ManagedSelector$SelectorProducer@1677891 execute
17/04/19 17:13:04.799 [qtp1978280845-15] DEBUG o.e.j.u.t.s.ExecuteProduceConsume - EPR Prod/org.eclipse.jetty.io.ManagedSelector$SelectorProducer@1677891 produce enter
17/04/19 17:13:04.799 [qtp1978280845-15] DEBUG o.e.j.u.t.s.ExecuteProduceConsume - EPR Prod/org.eclipse.jetty.io.ManagedSelector$SelectorProducer@1677891 producing
17/04/19 17:13:04.800 [qtp1978280845-15] DEBUG org.eclipse.jetty.io.ManagedSelector - Selector loop waiting on select
17/04/19 17:13:04.802 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7827ms org.eclipse.jetty.server.ServerConnector$ServerConnectorManager@31efc5b5
17/04/19 17:13:04.804 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/note, spark.ResponseTransformerRouteImpl$1@ebaa6cb
17/04/19 17:13:04.807 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/note, spark.ResponseTransformerRouteImpl$1@5fe1ce85
17/04/19 17:13:04.809 [Thread-1] DEBUG o.e.j.u.component.ContainerLifeCycle - ServerConnector@1a06506c{HTTP/1.1,[http/1.1]}{0.0.0.0:4000} added {acceptor-0@4c5a3f8a,POJO}
17/04/19 17:13:04.811 [Thread-1] DEBUG o.e.j.util.thread.QueuedThreadPool - queue acceptor-0@4c5a3f8a
17/04/19 17:13:04.812 [qtp1978280845-16] DEBUG o.e.j.util.thread.QueuedThreadPool - run acceptor-0@4c5a3f8a
17/04/19 17:13:04.813 [Thread-1] INFO  o.e.jetty.server.ServerConnector - Started ServerConnector@1a06506c{HTTP/1.1,[http/1.1]}{0.0.0.0:4000}
17/04/19 17:13:04.815 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7840ms ServerConnector@1a06506c{HTTP/1.1,[http/1.1]}{0.0.0.0:4000}
17/04/19 17:13:04.817 [Thread-1] INFO  org.eclipse.jetty.server.Server - Started @7841ms
17/04/19 17:13:04.817 [Thread-1] DEBUG o.e.j.u.component.AbstractLifeCycle - STARTED @7842ms org.eclipse.jetty.server.Server@2e01be57
17/04/19 17:13:04.817 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/status/jobs, spark.ResponseTransformerRouteImpl$1@2a640157
17/04/19 17:13:04.821 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/organization/:id, spark.ResponseTransformerRouteImpl$1@f80945f
17/04/19 17:13:04.822 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/organization, spark.ResponseTransformerRouteImpl$1@5ef6ae06
17/04/19 17:13:04.823 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/organization, spark.ResponseTransformerRouteImpl$1@1cdc4c27
17/04/19 17:13:04.824 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/organization/:id, spark.ResponseTransformerRouteImpl$1@1e8ce150
17/04/19 17:13:04.825 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/organization/:id, spark.ResponseTransformerRouteImpl$1@1b73be9f
17/04/19 17:13:04.902 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/agency/:id, spark.ResponseTransformerRouteImpl$1@232a7d73
17/04/19 17:13:04.903 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/agency, spark.RouteImpl$1@22ffa91a
17/04/19 17:13:04.904 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/agency, spark.ResponseTransformerRouteImpl$1@40844aab
17/04/19 17:13:04.906 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/agency, spark.ResponseTransformerRouteImpl$1@66746f57
17/04/19 17:13:04.908 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/agency/:id, spark.ResponseTransformerRouteImpl$1@28e8dde3
17/04/19 17:13:04.909 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/agency/:id/duplicate, spark.ResponseTransformerRouteImpl$1@1817f1eb
17/04/19 17:13:04.910 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/agency/:id/uploadbranding, spark.ResponseTransformerRouteImpl$1@301ec38b
17/04/19 17:13:04.911 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/agency/:id, spark.ResponseTransformerRouteImpl$1@c5dc4a2
17/04/19 17:13:04.915 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/calendar/:id, spark.ResponseTransformerRouteImpl$1@5669c5fb
17/04/19 17:13:04.915 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/calendar, spark.RouteImpl$1@5f9678e1
17/04/19 17:13:04.916 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/calendar, spark.ResponseTransformerRouteImpl$1@5a9d6f02
17/04/19 17:13:04.918 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/calendar, spark.ResponseTransformerRouteImpl$1@5ccbeb64
17/04/19 17:13:04.921 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/calendar/:id, spark.ResponseTransformerRouteImpl$1@3243b914
17/04/19 17:13:04.922 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/calendar/:id, spark.ResponseTransformerRouteImpl$1@626c44e7
17/04/19 17:13:04.926 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/route/:id, spark.ResponseTransformerRouteImpl$1@3016fd5e
17/04/19 17:13:04.927 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/route, spark.RouteImpl$1@53d102a2
17/04/19 17:13:04.928 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/route, spark.ResponseTransformerRouteImpl$1@5aac4250
17/04/19 17:13:04.929 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/route/merge, spark.ResponseTransformerRouteImpl$1@59f63e24
17/04/19 17:13:04.929 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/route, spark.ResponseTransformerRouteImpl$1@fade1fc
17/04/19 17:13:04.930 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/route/:id, spark.ResponseTransformerRouteImpl$1@613a8ee1
17/04/19 17:13:04.931 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/route/:id/uploadbranding, spark.ResponseTransformerRouteImpl$1@1b765a2c
17/04/19 17:13:04.932 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/route/:id, spark.ResponseTransformerRouteImpl$1@18920cc
17/04/19 17:13:04.935 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/routetype/:id, spark.ResponseTransformerRouteImpl$1@39a8312f
17/04/19 17:13:04.935 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/routetype, spark.RouteImpl$1@2c532cd8
17/04/19 17:13:04.936 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/routetype, spark.ResponseTransformerRouteImpl$1@3700ec9c
17/04/19 17:13:04.937 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/routetype, spark.ResponseTransformerRouteImpl$1@31000e60
17/04/19 17:13:04.939 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/routetype/:id, spark.ResponseTransformerRouteImpl$1@54c62d71
17/04/19 17:13:04.940 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/routetype/:id, spark.ResponseTransformerRouteImpl$1@2d1dee39
17/04/19 17:13:04.966 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/scheduleexception/:id, spark.ResponseTransformerRouteImpl$1@4af0df05
17/04/19 17:13:04.967 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/scheduleexception, spark.RouteImpl$1@acdb094
17/04/19 17:13:04.969 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/scheduleexception, spark.ResponseTransformerRouteImpl$1@2ceb80a1
17/04/19 17:13:04.971 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/scheduleexception, spark.ResponseTransformerRouteImpl$1@2072acb2
17/04/19 17:13:04.974 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/scheduleexception/:id, spark.ResponseTransformerRouteImpl$1@70a36a66
17/04/19 17:13:04.977 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/scheduleexception/:id, spark.ResponseTransformerRouteImpl$1@d3957fe
17/04/19 17:13:04.985 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/stop/:id, spark.ResponseTransformerRouteImpl$1@158a3b2e
17/04/19 17:13:04.985 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/stop, spark.RouteImpl$1@5e1fa5b1
17/04/19 17:13:04.986 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/stop, spark.ResponseTransformerRouteImpl$1@3e821657
17/04/19 17:13:04.987 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/stop/mergeStops, spark.ResponseTransformerRouteImpl$1@615091b8
17/04/19 17:13:04.988 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/stop, spark.ResponseTransformerRouteImpl$1@43b6123e
17/04/19 17:13:04.989 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/stop/:id, spark.ResponseTransformerRouteImpl$1@12bd8a64
17/04/19 17:13:04.990 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/stop/:id, spark.ResponseTransformerRouteImpl$1@ebb6851
17/04/19 17:13:04.997 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/trip/:id, spark.ResponseTransformerRouteImpl$1@3a6f2de3
17/04/19 17:13:04.997 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/trip, spark.RouteImpl$1@56303b57
17/04/19 17:13:04.999 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/trip, spark.ResponseTransformerRouteImpl$1@78f5c518
17/04/19 17:13:05.000 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/trip, spark.ResponseTransformerRouteImpl$1@753432a2
17/04/19 17:13:05.001 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/trip/:id, spark.ResponseTransformerRouteImpl$1@44e3a2b2
17/04/19 17:13:05.002 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/trip/:id, spark.ResponseTransformerRouteImpl$1@1d2bd371
17/04/19 17:13:05.015 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/trippattern/:id, spark.ResponseTransformerRouteImpl$1@26bab2f1
17/04/19 17:13:05.017 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/trippattern, spark.RouteImpl$1@68ead359
17/04/19 17:13:05.018 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/trippattern, spark.ResponseTransformerRouteImpl$1@a43ce46
17/04/19 17:13:05.019 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/trippattern, spark.ResponseTransformerRouteImpl$1@37052337
17/04/19 17:13:05.020 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/trippattern/:id, spark.ResponseTransformerRouteImpl$1@4d722ac9
17/04/19 17:13:05.021 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/trippattern/:id, spark.ResponseTransformerRouteImpl$1@3d97a632
17/04/19 17:13:05.030 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/snapshot/:id, spark.ResponseTransformerRouteImpl$1@60957c0f
17/04/19 17:13:05.031 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/snapshot, spark.RouteImpl$1@fcb4004
17/04/19 17:13:05.032 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/snapshot, spark.ResponseTransformerRouteImpl$1@2dd29a59
17/04/19 17:13:05.034 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/snapshot, spark.ResponseTransformerRouteImpl$1@40dff0b7
17/04/19 17:13:05.035 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/snapshot/import, spark.ResponseTransformerRouteImpl$1@1253e7cb
17/04/19 17:13:05.036 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/snapshot/:id, spark.ResponseTransformerRouteImpl$1@3b0c9195
17/04/19 17:13:05.037 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/snapshot/:id/restore, spark.ResponseTransformerRouteImpl$1@302a07d
17/04/19 17:13:05.042 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/snapshot/:id/downloadtoken, spark.ResponseTransformerRouteImpl$1@7e8dcdaa
17/04/19 17:13:05.045 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/snapshot/:id, spark.ResponseTransformerRouteImpl$1@5c08c46a
17/04/19 17:13:05.048 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/downloadsnapshot/:token, spark.RouteImpl$1@708400f6
17/04/19 17:13:05.055 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedinfo/:id, spark.ResponseTransformerRouteImpl$1@c35172e
17/04/19 17:13:05.056 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/feedinfo, spark.RouteImpl$1@3cc41abc
17/04/19 17:13:05.058 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/feedinfo, spark.ResponseTransformerRouteImpl$1@3668d4
17/04/19 17:13:05.059 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/feedinfo/:id, spark.ResponseTransformerRouteImpl$1@7f6f61c8
17/04/19 17:13:05.060 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/feedinfo/:id, spark.ResponseTransformerRouteImpl$1@4097cac
17/04/19 17:13:05.064 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/fare/:id, spark.ResponseTransformerRouteImpl$1@52d645b1
17/04/19 17:13:05.065 [main] DEBUG spark.route.Routes - Adds route: options, /api/manager/secure/fare, spark.RouteImpl$1@2cc3ad05
17/04/19 17:13:05.068 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/fare, spark.ResponseTransformerRouteImpl$1@3d9f6567
17/04/19 17:13:05.069 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/fare, spark.ResponseTransformerRouteImpl$1@35e5d0e5
17/04/19 17:13:05.075 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/fare/:id, spark.ResponseTransformerRouteImpl$1@655ef322
17/04/19 17:13:05.076 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/fare/:id, spark.ResponseTransformerRouteImpl$1@5066d65f
17/04/19 17:13:05.101 [main] WARN  c.c.d.m.c.api.GtfsApiController - No extension provided for GTFS API
17/04/19 17:13:05.101 [main] WARN  c.c.datatools.manager.DataManager - Config property modules.gtfsapi.update_frequency not found
17/04/19 17:13:05.103 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//hello, spark.RouteImpl$1@20b12f8a
17/04/19 17:13:05.105 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//feeds, spark.ResponseTransformerRouteImpl$1@522a32b1
17/04/19 17:13:05.107 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//feeds/:id, spark.ResponseTransformerRouteImpl$1@5e01a982
17/04/19 17:13:05.110 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//routes, spark.ResponseTransformerRouteImpl$1@58fb7731
17/04/19 17:13:05.110 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//routes/:id, spark.ResponseTransformerRouteImpl$1@3fb6cf60
17/04/19 17:13:05.112 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//stops, spark.ResponseTransformerRouteImpl$1@5e8ac0e1
17/04/19 17:13:05.113 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//stops/:id, spark.ResponseTransformerRouteImpl$1@6955cb39
17/04/19 17:13:05.114 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//trips, spark.ResponseTransformerRouteImpl$1@6b7906b3
17/04/19 17:13:05.115 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//trips/:id, spark.ResponseTransformerRouteImpl$1@395b56bb
17/04/19 17:13:05.118 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//trips/:id/stoptimes, spark.ResponseTransformerRouteImpl$1@68fa0ba8
17/04/19 17:13:05.120 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//patterns, spark.ResponseTransformerRouteImpl$1@64a8c844
17/04/19 17:13:05.121 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//, spark.RouteImpl$1@52de51b6
17/04/19 17:13:05.127 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//graphql, spark.ResponseTransformerRouteImpl$1@2e9fda69
17/04/19 17:13:05.128 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager//graphql, spark.ResponseTransformerRouteImpl$1@1755e85b
17/04/19 17:13:05.130 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager//graphql/schema, spark.ResponseTransformerRouteImpl$1@2371aaca
17/04/19 17:13:05.133 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager//graphql/schema, spark.ResponseTransformerRouteImpl$1@63fdab07
17/04/19 17:13:05.133 [main] WARN  c.c.datatools.manager.DataManager - Config property modules.gtfsplus.enabled not found
17/04/19 17:13:05.133 [main] WARN  c.c.datatools.manager.DataManager - Config property modules.gtfsplus.enabled not found
17/04/19 17:13:05.133 [main] WARN  c.c.datatools.manager.DataManager - Config property modules.gtfsplus.enabled not found
17/04/19 17:13:05.147 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/user/:id, spark.ResponseTransformerRouteImpl$1@3f390d63
17/04/19 17:13:05.147 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/user/:id/recentactivity, spark.ResponseTransformerRouteImpl$1@2374d36a
17/04/19 17:13:05.148 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/user, spark.ResponseTransformerRouteImpl$1@47a5b70d
17/04/19 17:13:05.149 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/secure/usercount, spark.ResponseTransformerRouteImpl$1@38f116f6
17/04/19 17:13:05.150 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/secure/user, spark.ResponseTransformerRouteImpl$1@5c530d1e
17/04/19 17:13:05.152 [main] DEBUG spark.route.Routes - Adds route: put, /api/manager/secure/user/:id, spark.ResponseTransformerRouteImpl$1@c5ee75e
17/04/19 17:13:05.153 [main] DEBUG spark.route.Routes - Adds route: delete, /api/manager/secure/user/:id, spark.ResponseTransformerRouteImpl$1@70efb718
17/04/19 17:13:05.157 [main] DEBUG spark.route.Routes - Adds route: post, /api/manager/public/user, spark.ResponseTransformerRouteImpl$1@4e858e0a
17/04/19 17:13:05.158 [main] WARN  c.c.datatools.manager.DataManager - Config property modules.dump.enabled not found
17/04/19 17:13:05.159 [main] WARN  c.c.datatools.manager.DataManager - Config property modules.dump.enabled not found
17/04/19 17:13:05.159 [main] WARN  c.c.datatools.manager.DataManager - Config property modules.dump.enabled not found
17/04/19 17:13:05.161 [main] DEBUG spark.route.Routes - Adds route: before, /api/manager/secure/*, spark.FilterImpl$1@b7838a9
17/04/19 17:13:05.162 [main] DEBUG spark.route.Routes - Adds route: after, /api/manager/*, spark.FilterImpl$1@5733f295
17/04/19 17:13:05.163 [main] DEBUG spark.route.Routes - Adds route: get, /api/manager/*, spark.RouteImpl$1@47a86fbb
17/04/19 17:13:05.163 [main] DEBUG spark.route.Routes - Adds route: get, /*, spark.RouteImpl$1@19553973
17/04/19 17:13:05.164 [main] WARN  c.c.datatools.manager.DataManager - Config property extensions.mtc.enabled not found
17/04/19 17:13:05.164 [main] WARN  c.c.datatools.manager.DataManager - Config property extensions.mtc.enabled not found
17/04/19 17:13:05.164 [main] WARN  c.c.datatools.manager.DataManager - Config property extensions.mtc.enabled not found
17/04/19 17:13:05.164 [main] INFO  c.c.datatools.manager.DataManager - Registering TransitLand Resource
17/04/19 17:13:05.165 [main] INFO  c.c.datatools.manager.DataManager - Registering TransitFeeds Resource
17/04/19 17:13:05.167 [main] INFO  com.conveyal.datatools.LoadFeedTest - ProcessGtfsSnapshotMergeTest setup
17/04/19 17:13:05.169 [main] INFO  c.c.d.manager.persistence.DataStore - /private/tmp/feedsources
17/04/19 17:13:05.295 [main] INFO  c.c.d.manager.persistence.DataStore - /private/tmp/feedversions
17/04/19 17:13:05.389 [main] INFO  c.c.d.manager.persistence.FeedStore - Writing file to /tmp/test-20170419T171305+02-f19c9daa-3bf3-4299-92a3-559a095fa45b.zip
17/04/19 17:13:05.403 [main] INFO  c.c.d.manager.persistence.FeedStore - Copying version to latest <FeedSource test (f19c9daa-3bf3-4299-92a3-559a095fa45b)>
17/04/19 17:13:05.426 [main] INFO  c.c.d.manager.models.FeedVersion - New GTFS file saved: test-20170419T171305+02-f19c9daa-3bf3-4299-92a3-559a095fa45b.zip
17/04/19 17:13:05.447 [main] INFO  c.c.d.e.ProcessGtfsSnapshotMergeTest - ProcessGtfsSnapshotMergeTest setup
17/04/19 17:13:05.655 [main] INFO  c.c.d.e.j.ProcessGtfsSnapshotMerge - GTFS Snapshot Merge for feedVersion test-20170419T171305+02-f19c9daa-3bf3-4299-92a3-559a095fa45b.zip
17/04/19 17:13:06.133 [main] INFO  c.c.d.e.j.ProcessGtfsSnapshotMerge - Cleared old data
17/04/19 17:13:06.134 [main] WARN  com.conveyal.gtfs.BaseGTFSCache - Feed test-20170419T171305+02-f19c9daa-3bf3-4299-92a3-559a095fa45b not found locally
java.lang.NullPointerException
	at com.conveyal.datatools.manager.models.FeedVersion.getGtfsFeed(FeedVersion.java:182)
	at com.conveyal.datatools.editor.jobs.ProcessGtfsSnapshotMerge.run(ProcessGtfsSnapshotMerge.java:130)
	at com.conveyal.datatools.editor.ProcessGtfsSnapshotMergeTest.setUp(ProcessGtfsSnapshotMergeTest.java:39)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Tests run: 4, Failures: 4, Errors: 0, Skipped: 0, Time elapsed: 9.015 sec <<< FAILURE!

Fix deployment documentation

Issue by l-vincent-l
Tuesday Jul 11, 2017 at 13:34 GMT
Originally opened as catalogueglobal#23

At the page :
http://conveyal-data-tools.readthedocs.io/en/dev/dev/deployment/

It is said you have to run :
$ java -jar target/dt-v1.0.0.jar /path/to/env.yml /path/to/server.yml
to build the java package, I think you have to run :
$ mvn package -DskipTests

Run application with no auth for development

Issue by landonreed
Monday Jun 12, 2017 at 20:59 GMT
Originally opened as catalogueglobal#19

If AUTH0_CLIENT_ID config property is null, when check user is called, attach a mock user to the request (where the user would normally be constructed from the Auth0 token).
When any permission check is performed, short circuit this check (return true for application-admin permission).

landonreed included the following code: https://github.com/catalogueglobal/datatools-server/pull/19/commits

Use GTFS API to write files to local file system

Issue by landonreed
Thursday Apr 20, 2017 at 17:09 GMT
Originally opened as catalogueglobal#12

Use ApiMain::registerFeedSource to write feed version files to local file system.

https://github.com/conveyal/analysis-backend/blob/master/src/main/java/com/conveyal/taui/controllers/BundleController.java#L121

Trip pattern direction not saved properly

Issue by landonreed
Wednesday Sep 20, 2017 at 19:42 GMT
Originally opened as catalogueglobal#41

When updating the trip pattern's direction using the UI, the direction value does not appear to be updating properly. The root cause appears to be that the TripPatternSerializer::serialize method is not being called properly for some reason (this happens in Base.java).

Nevertheless, it's not entirely clear that we even need this serializer. There is no issue in just delivering these values to the client as A and B as they are specified in the TripDirection enum (just needs a corresponding fix on the client).

HTTP request synchronization behind security proxies

Issue by zbouziane
Tuesday Mar 07, 2017 at 11:22 GMT
Originally opened as catalogueglobal#1

Some actions need two http successive requests to be performed. Downloading a feed is an example of such an action. The first request download a token '/api/manager/${SecureOrPublic}/feedversion/${FeedVersion.id}/downloadtoken' then the second request uses this token to download the feed /api/manager/downloadfeed/${DownloadedToken}. The token is available for only one dowload, which causes the download of feeds fails for clients behind a security proxy that plays any http request twice. A first time to test the server and a second time to serve the original requester.

Server not starting on java -jar target/dt-6a27c88-dirty.jar configurations/default/env.yml configurations/default/server.yml

when runing last command

java -jar target/dt-6a27c88-dirty.jar configurations/default/env.yml configurations/default/server.yml

it stops here

ubuntu@ip-172-31-3-112:~/datatools-server$ java -jar target/dt-6a27c88-dirty.jar configurations/default/env.yml configurations/default/server.yml
15:06:07.311 [main] INFO c.c.datatools.manager.DataManager - Loading env.yml: configurations/default/env.yml
15:06:07.314 [main] INFO c.c.datatools.manager.DataManager - Loading server.yml: configurations/default/server.yml
15:06:07.418 [main] INFO c.c.d.manager.persistence.DataStore - /home/ubuntu/datatools-server/data/mapdb/projects
15:06:07.569 [main] WARN c.c.datatools.manager.DataManager - Config property application.data.gtfs_s3_bucket not found
15:06:07.569 [main] WARN c.c.datatools.manager.DataManager - Config property application.data.gtfs_s3_bucket not found
15:06:07.571 [main] WARN c.c.datatools.manager.DataManager - Config property application.data.s3_credentials_file not found
15:06:07.572 [main] WARN c.c.datatools.manager.DataManager - Config property application.data.s3_credentials_file not found
15:06:07.572 [main] INFO c.c.datatools.manager.DataManager - Initializing gtfs cache locally (no s3 bucket) /home/ubuntu/datatools-server/data/gtfs
log4j:WARN No appenders could be found for logger (com.amazonaws.AmazonWebServiceClient).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
15:06:08.324 [main] INFO com.conveyal.gtfs.GTFSCache - No bucket specified; GTFS Cache will run locally
15:06:08.495 [Thread-1] INFO org.eclipse.jetty.util.log - Logging initialized @1736ms
15:06:08.637 [Thread-1] INFO s.e.jetty.EmbeddedJettyServer - == Spark has ignited ...
15:06:08.638 [Thread-1] INFO s.e.jetty.EmbeddedJettyServer - >> Listening on 0.0.0.0:4000
15:06:08.641 [Thread-1] INFO org.eclipse.jetty.server.Server - jetty-9.3.z-SNAPSHOT
15:06:08.749 [Thread-1] INFO o.e.jetty.server.ServerConnector - Started ServerConnector@9fa409b{HTTP/1.1,[http/1.1]}{0.0.0.0:4000}
15:06:08.750 [Thread-1] INFO org.eclipse.jetty.server.Server - Started @1999ms
15:06:08.955 [main] WARN c.c.d.m.c.api.GtfsApiController - No extension provided for GTFS API
15:06:08.967 [pool-1-thread-1] INFO c.c.d.manager.jobs.FeedUpdater - Fetching feeds...
15:06:08.968 [pool-1-thread-1] INFO c.c.d.manager.jobs.FeedUpdater - Current eTag list {}
15:06:08.976 [main] WARN c.c.datatools.manager.DataManager - Config property modules.gtfsplus.enabled not found
15:06:08.976 [main] WARN c.c.datatools.manager.DataManager - Config property modules.gtfsplus.enabled not found
15:06:08.976 [main] WARN c.c.datatools.manager.DataManager - Config property modules.gtfsplus.enabled not found
15:06:08.990 [main] WARN c.c.datatools.manager.DataManager - Config property modules.dump.enabled not found
15:06:08.991 [main] WARN c.c.datatools.manager.DataManager - Config property modules.dump.enabled not found
15:06:08.991 [main] WARN c.c.datatools.manager.DataManager - Config property modules.dump.enabled not found
15:06:08.993 [main] WARN c.c.datatools.manager.DataManager - Config property extensions.mtc.enabled not found
15:06:08.994 [main] WARN c.c.datatools.manager.DataManager - Config property extensions.mtc.enabled not found
15:06:08.994 [main] WARN c.c.datatools.manager.DataManager - Config property extensions.mtc.enabled not found
15:06:08.999 [main] INFO c.c.datatools.manager.DataManager - Registering TransitLand Resource
15:06:09.000 [main] INFO c.c.datatools.manager.DataManager - Registering TransitFeeds Resource

Auth refactor into mongo-pojo

Issue by landonreed
Wednesday Sep 20, 2017 at 18:12 GMT
Originally opened as catalogueglobal#40

Merge approved auth refactor (#24) into MongoDB/RDBMS branch for staging testing.

landonreed included the following code: https://github.com/catalogueglobal/datatools-server/pull/40/commits

Fail of Uploading In Local Storage Mode

Issue by zbouziane
Thursday Apr 20, 2017 at 10:37 GMT
Originally opened as catalogueglobal#11

With use_s3_storage set to false:

17/04/20 12:14:10.621 [Thread-14] WARN  com.conveyal.gtfs.BaseGTFSCache - Feed tampere-20170420T121408+02-f6a79e00-b150-4725-b385-712826a46351 not found locally
java.lang.NullPointerException
	at com.conveyal.datatools.manager.models.FeedVersion.getGtfsFeed(FeedVersion.java:182)
	at com.conveyal.datatools.editor.jobs.ProcessGtfsSnapshotMerge.run(ProcessGtfsSnapshotMerge.java:130)
	at java.lang.Thread.run(Thread.java:745)

Extensions refactor

Issue by landonreed
Tuesday Aug 29, 2017 at 15:21 GMT
Originally opened as catalogueglobal#32

Refactor the extensions interface so that common functionality is contained within default methods. This improves the modularity of the code and allows for other extensions to be added with relatively little boilerplate/effort and much enhanced clarity.

landonreed included the following code: https://github.com/catalogueglobal/datatools-server/pull/32/commits

Unable to run `mvn package`

When running mvn package
I have this error message

Running com.conveyal.datatools.editor.ProcessGtfsSnapshotMergeTest
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/vincent/.m2/repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/vincent/.m2/repository/org/slf4j/slf4j-simple/1.7.10/slf4j-simple-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
14:36:49.035 [main] INFO  com.conveyal.datatools.DatatoolsTest - DatatoolsTest setup
14:36:49.170 [main] INFO  c.c.datatools.manager.DataManager - Loading env.yml: ../configurations/gtfs.works/dev/settings.yml
14:36:49.171 [main] INFO  c.c.datatools.manager.DataManager - Loading server.yml: ../configurations/gtfs.works/dev/server.yml
java.io.FileNotFoundException: ../configurations/gtfs.works/dev/settings.yml (Aucun fichier ou dossier de ce type)
	at java.io.FileInputStream.open0(Native Method)
	at java.io.FileInputStream.open(FileInputStream.java:195)
	at java.io.FileInputStream.<init>(FileInputStream.java:138)
	at com.conveyal.datatools.manager.DataManager.loadConfig(DataManager.java:307)
	at com.conveyal.datatools.manager.DataManager.main(DataManager.java:78)
	at com.conveyal.datatools.DatatoolsTest.setUp(DatatoolsTest.java:25)
	at com.conveyal.datatools.LoadFeedTest.setUp(LoadFeedTest.java:28)
	at com.conveyal.datatools.editor.ProcessGtfsSnapshotMergeTest.setUp(ProcessGtfsSnapshotMergeTest.java:35)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)

fix(editor): fix memory leaks on editor db transactions

Issue by landonreed
Tuesday May 02, 2017 at 16:02 GMT
Originally opened as catalogueglobal#16

A number of model classes had getters that created transactions that were never rolled back. This
moves those into addDerivedInfo functions.

fixes #14

landonreed included the following code: https://github.com/catalogueglobal/datatools-server/pull/16/commits

The right filename for downloaded feeds

Issue by zbouziane
Tuesday Mar 07, 2017 at 17:13 GMT
Originally opened as catalogueglobal#3

The download request '/api/manager/downloadfeed/${token}' when succeeding return a zip file with a randomly generated name. In the response header there is "Content-Disposition: attachment; filename=test4474876194626471181.zip". Is it possible to return the right filename instead?

Get upstream

Issue by l-vincent-l
Friday Sep 01, 2017 at 14:20 GMT
Originally opened as catalogueglobal#33

l-vincent-l included the following code: https://github.com/catalogueglobal/datatools-server/pull/33/commits

Auth refactor

Issue by evansiroky
Tuesday Jul 25, 2017 at 01:04 GMT
Originally opened as catalogueglobal#24

Add a server-side route for auth0's silent authentication.

This PR is required in order for catalogueglobal/datatools-ui#12 to work properly.

evansiroky included the following code: https://github.com/catalogueglobal/datatools-server/pull/24/commits

Fix download merged project feed

Issue by landonreed
Tuesday Sep 19, 2017 at 19:31 GMT
Originally opened as catalogueglobal#37

Problem

The "download merged feed for project" feature previously was run synchronously within the API controller rather than with the "heavy executor" designed to handle large jobs asynchronously. Recently, we discovered with NYSDOT that this method was not performing properly (presumably because of the large number of feeds included therein).

Solution

Nothing fancy. This PR moves the merge feed code into a MonitorableJob so that the UI will receive progress updates and is accompanied by a UI PR at catalogueglobal/datatools-ui#27. It also creates some additional controllers to handle the token creation/delivery to the UI (modeled after FeedVersion and Snapshot downloads).

landonreed included the following code: https://github.com/catalogueglobal/datatools-server/pull/37/commits

DB transaction left unclosed

Issue by zbouziane
Friday Apr 28, 2017 at 13:00 GMT
Originally opened as catalogueglobal#14

...
09:10:36.239 [Finalizer] ERROR c.c.d.editor.datastore.DatabaseTx - DB transaction left unclosed, this signifies a memory leak!
...

Update VEX URL

HTTP Status Code 404 for Recent Activity

Issue by zbouziane
Tuesday Apr 18, 2017 at 09:56 GMT
Originally opened as catalogueglobal#7

11:43:49.287 [qtp83546919-15] ERROR c.c.datatools.manager.DataManager - error
java.lang.NullPointerException: null
at com.conveyal.datatools.manager.controllers.api.UserController.getRecentActivity(UserController.java:215) ~[dt-6a27c88-dirty.jar:na]
at spark.ResponseTransformerRouteImpl$1.handle(ResponseTransformerRouteImpl.java:47) ~[dt-6a27c88-dirty.jar:na]
at spark.http.matching.Routes.execute(Routes.java:61) ~[dt-6a27c88-dirty.jar:na]
at spark.http.matching.MatcherFilter.doFilter(MatcherFilter.java:126) ~[dt-6a27c88-dirty.jar:na]
at spark.embeddedserver.jetty.JettyHandler.doHandle(JettyHandler.java:50) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:189) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.server.Server.handle(Server.java:517) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) ~[dt-6a27c88-dirty.jar:na]
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) ~[dt-6a27c88-dirty.jar:na]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_112]
11:43:49.288 [qtp83546919-15] INFO spark.http.matching.MatcherFilter - The requested route [/api/manager/secure/user/auth0|57a1f0c5b8155e90194e5087/recentactivity] has not been mapped in Spark

feat(cors): enable cors/options preflight for all API routes

Issue by landonreed
Tuesday Apr 04, 2017 at 15:20 GMT
Originally opened as catalogueglobal#5

@zbouziane here's an update for cors to fix #2.

This adds config properties to customize the cors origins, headers, and methods. It also adds SparkUtils.enableCors and cleans up some lingering code in DataManager.

landonreed included the following code: https://github.com/catalogueglobal/datatools-server/pull/5/commits

Server responding Bad Request for GraphQL request

Issue by zbouziane
Thursday Sep 14, 2017 at 12:59 GMT
Originally opened as catalogueglobal#34

On mongo-pojo branch at commit number 7b1b4d8e30c81b8ffe0d6dfeaaedc4f3708b5e77:

The processes of creating a project, creating a feed source and creating a feed version through an upload of GTFS zipped file have all succeeded. The two databases, postgresql and mongo, have now the corresponding data. Then in the Version summary of the UI I see 0 Agency Count, 0 Route Count and 0 Tip Count. This is not true! None of the corresponding table is empty.
Then when I switch to any of the tabs Routes, Patterns or Stops, nothing happens on the UI except the "waiting animation".

The javascript console displays the following:

routes.js:34 GET http://localhost:4000/api/manager/graphql?query=%0Aquery%20routeQuery(%24feedId%3A%20%5BString%5D)%20%7B%0A%20%20routes(feed_id%3A%20%24feedId)%20%7B%0A%20%20%20%20route_id%0A%20%20%20%20route_short_name%0A%20%20%20%20route_long_name%2C%0A%20%20%20%20route_desc%2C%0A%20%20%20%20route_url%2C%0A%20%20%20%20trip_count%2C%0A%20%20%20%20pattern_count%0A%20%20%7D%0A%7D%0A&variables=%7B%22feedId%22%3A%22TISSEO-20170914T142615-02-bc15c0a0-d47e-4b20-8d51-87537956c281%22%7D 400 (Bad Request)
(anonymous) @ routes.js:34
(anonymous) @ index.js:11
(anonymous) @ promise.js:16
(anonymous) @ multi.js:12
(anonymous) @ fetch.js:70
(anonymous) @ middleware.js:22
onComponentMount @ Routes.js:18
value @ RouteLayout.js:15
performInitialMount @ ReactCompositeComponent.js:351
mountComponent @ ReactCompositeComponent.js:258
mountComponent @ ReactReconciler.js:46
performInitialMount @ ReactCompositeComponent.js:371
mountComponent @ ReactCompositeComponent.js:258
mountComponent @ ReactReconciler.js:46
mountChildren @ ReactMultiChild.js:238
_createInitialChildren @ ReactDOMComponent.js:697
mountComponent @ ReactDOMComponent.js:516
mountComponent @ ReactReconciler.js:46
_updateRenderedComponent @ ReactCompositeComponent.js:765
_performComponentUpdate @ ReactCompositeComponent.js:724
updateComponent @ ReactCompositeComponent.js:645
receiveComponent @ ReactCompositeComponent.js:547
receiveComponent @ ReactReconciler.js:125
_updateRenderedComponent @ ReactCompositeComponent.js:754
_performComponentUpdate @ ReactCompositeComponent.js:724
updateComponent @ ReactCompositeComponent.js:645
receiveComponent @ ReactCompositeComponent.js:547
receiveComponent @ ReactReconciler.js:125
_updateRenderedComponent @ ReactCompositeComponent.js:754
_performComponentUpdate @ ReactCompositeComponent.js:724
updateComponent @ ReactCompositeComponent.js:645
receiveComponent @ ReactCompositeComponent.js:547
receiveComponent @ ReactReconciler.js:125
_updateRenderedComponent @ ReactCompositeComponent.js:754
_performComponentUpdate @ ReactCompositeComponent.js:724
updateComponent @ ReactCompositeComponent.js:645
receiveComponent @ ReactCompositeComponent.js:547
receiveComponent @ ReactReconciler.js:125
updateChildren @ ReactChildReconciler.js:109
_reconcilerUpdateChildren @ ReactMultiChild.js:213
_updateChildren @ ReactMultiChild.js:312
updateChildren @ ReactMultiChild.js:299
_updateDOMChildren @ ReactDOMComponent.js:936
updateComponent @ ReactDOMComponent.js:754
receiveComponent @ ReactDOMComponent.js:716
receiveComponent @ ReactReconciler.js:125
_updateRenderedComponent @ ReactCompositeComponent.js:754
_performComponentUpdate @ ReactCompositeComponent.js:724
updateComponent @ ReactCompositeComponent.js:645
performUpdateIfNecessary @ ReactCompositeComponent.js:561
performUpdateIfNecessary @ ReactReconciler.js:157
runBatchedUpdates @ ReactUpdates.js:150
perform @ Transaction.js:140
perform @ Transaction.js:140
perform @ ReactUpdates.js:89
flushBatchedUpdates @ ReactUpdates.js:172
closeAll @ Transaction.js:206
perform @ Transaction.js:153
batchedUpdates @ ReactDefaultBatchingStrategy.js:62
enqueueUpdate @ ReactUpdates.js:200
enqueueUpdate @ ReactUpdateQueue.js:24
enqueueCallback @ ReactUpdateQueue.js:123
ReactComponent.setState @ ReactComponent.js:65
value @ Transition.js:181
(anonymous) @ Transition.js:155
nextCallback @ Transition.js:195
routes.js:39 Uncaught (in promise) TypeError: Cannot read property 'length' of undefined
    at reducer (routes.js:39)
    at combineReducers.js:132
    at combineReducers.js:132
    at c (createStore.js:179)
    at index.js:14
    at promise.js:16
    at multi.js:12
    at fetch.js:70
    at middleware.js:22
    at dispatch (applyMiddleware.js:45)

and in the datatools-server logs displays the following:

sept. 14 14:30:10.511 [qtp355582964-19] INFO  graphql.GraphQL - Executing request. operation name: null. Request:
query routeQuery($feedId: [String]) {
  routes(feed_id: $feedId) {
    route_id
    route_short_name
    route_long_name,
    route_desc,
    route_url,
    trip_count,
    pattern_count
  }
}

Unable to run mvn package

When running mvn package
I have this error message

Running com.conveyal.datatools.editor.ProcessGtfsSnapshotMergeTest
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/vincent/.m2/repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/vincent/.m2/repository/org/slf4j/slf4j-simple/1.7.10/slf4j-simple-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
14:36:49.035 [main] INFO  com.conveyal.datatools.DatatoolsTest - DatatoolsTest setup
14:36:49.170 [main] INFO  c.c.datatools.manager.DataManager - Loading env.yml: ../configurations/gtfs.works/dev/settings.yml
14:36:49.171 [main] INFO  c.c.datatools.manager.DataManager - Loading server.yml: ../configurations/gtfs.works/dev/server.yml
java.io.FileNotFoundException: ../configurations/gtfs.works/dev/settings.yml (Aucun fichier ou dossier de ce type)
	at java.io.FileInputStream.open0(Native Method)
	at java.io.FileInputStream.open(FileInputStream.java:195)
	at java.io.FileInputStream.<init>(FileInputStream.java:138)
	at com.conveyal.datatools.manager.DataManager.loadConfig(DataManager.java:307)
	at com.conveyal.datatools.manager.DataManager.main(DataManager.java:78)
	at com.conveyal.datatools.DatatoolsTest.setUp(DatatoolsTest.java:25)
	at com.conveyal.datatools.LoadFeedTest.setUp(LoadFeedTest.java:28)
	at com.conveyal.datatools.editor.ProcessGtfsSnapshotMergeTest.setUp(ProcessGtfsSnapshotMergeTest.java:35)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)

Mismatched trip stopTimes and pattern stops encountered

Issue by landonreed
Monday Sep 25, 2017 at 17:37 GMT
Originally opened as catalogueglobal#43

It appears that in a recent refactor, an important step in validating that the trip stopTimes match the pattern stops was ignored when creating new trips. The step that confirms matching stop IDs between these two sets needs to be added to TripController::validateTrip.

Exception raised due to missing directory

Issue by cgalant
Tuesday Sep 26, 2017 at 13:30 GMT
Originally opened as catalogueglobal#45

When deploying the datatools-server for the first time, and if you put different directories for each properties requiring a path on the disk filesystem you got the error below when importing a new feedversion. This is due to a missing directory not created by the datatools on first hand.

Fix proposal : check that the directory exists and create it otherwise.

14:48:31.250 [qtp1641596316-19] WARN org.eclipse.jetty.server.HttpChannel - //localhost:4200/api/manager/secure/feedversion?feedSourceId=07b8812b-bce6-40b1-b72a-f3324bbe837a&lastModified=1503075401000 java.lang.ExceptionInInitializerError: null at com.conveyal.datatools.manager.controllers.api.FeedVersionController.createFeedVersion(FeedVersionController.java:112) ~[classes/:na] at spark.ResponseTransformerRouteImpl$1.handle(ResponseTransformerRouteImpl.java:47) ~[spark-core-2.5.jar:na] at spark.http.matching.Routes.execute(Routes.java:61) ~[spark-core-2.5.jar:na] at spark.http.matching.MatcherFilter.doFilter(MatcherFilter.java:126) ~[spark-core-2.5.jar:na] at spark.embeddedserver.jetty.JettyHandler.doHandle(JettyHandler.java:50) ~[spark-core-2.5.jar:na] at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:189) ~[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119) ~[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.server.Server.handle(Server.java:517) ~[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) ~[jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242) [jetty-server-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261) [jetty-io-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) [jetty-io-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75) [jetty-io-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213) [jetty-util-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147) [jetty-util-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) [jetty-util-9.3.6.v20151106.jar:9.3.6.v20151106] at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) [jetty-util-9.3.6.v20151106.jar:9.3.6.v20151106] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] Caused by: java.lang.IllegalArgumentException: Not a directory or not found: D:\tmp\data\gtfs at com.conveyal.datatools.manager.persistence.FeedStore.getPath(FeedStore.java:95) ~[classes/:na] at com.conveyal.datatools.manager.persistence.FeedStore.<init>(FeedStore.java:66) ~[classes/:na] at com.conveyal.datatools.manager.persistence.FeedStore.<init>(FeedStore.java:54) ~[classes/:na] at com.conveyal.datatools.manager.models.FeedVersion.<clinit>(FeedVersion.java:81) ~[classes/:na] ... 19 common frames omitted

Implement shapefile export for routes/stops

Issue by landonreed
Friday May 26, 2017 at 14:12 GMT
Originally opened as catalogueglobal#17

This feature was originally part of the https://github.com/conveyal/gtfs-editor but during the major refactor of that application the feature was lost.

Test failure

Issue by zbouziane
Wednesday Apr 19, 2017 at 14:10 GMT
Originally opened as catalogueglobal#9

$ mvn test
...
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.conveyal.datatools.editor.ProcessGtfsSnapshotMergeTest
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/zbouziane/.m2/repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/zbouziane/.m2/repository/org/slf4j/slf4j-simple/1.7.10/slf4j-simple-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
16:03:53,041 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
16:03:53,041 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
16:03:53,042 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [file:/Users/zbouziane/tmp/tmp/datatools-server/target/classes/logback.xml]
16:03:53,044 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs multiple times on the classpath.
16:03:53,044 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/Users/zbouziane/.m2/repository/com/conveyal/osm-lib/1.1.0-SNAPSHOT/osm-lib-1.1.0-SNAPSHOT.jar!/logback.xml]
16:03:53,044 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [file:/Users/zbouziane/tmp/tmp/datatools-server/target/classes/logback.xml]
16:03:53,158 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
16:03:58,171 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
16:03:58,178 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
16:03:58,203 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
16:03:58,254 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to DEBUG
16:03:58,254 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
16:03:58,254 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
16:03:58,255 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@22eeefeb - Registering current configuration as safe fallback point

SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
17/04/19 16:03:58.263 [main] INFO  com.conveyal.datatools.DatatoolsTest - DatatoolsTest setup
17/04/19 16:03:58.461 [main] INFO  c.c.datatools.manager.DataManager - Loading env.yml: ../configurations/gtfs.works/dev/settings.yml
17/04/19 16:03:58.462 [main] INFO  c.c.datatools.manager.DataManager - Loading server.yml: ../configurations/gtfs.works/dev/server.yml
java.io.FileNotFoundException: ../configurations/gtfs.works/dev/settings.yml (No such file or directory)
	at java.io.FileInputStream.open0(Native Method)
	at java.io.FileInputStream.open(FileInputStream.java:195)
	at java.io.FileInputStream.<init>(FileInputStream.java:138)
	at com.conveyal.datatools.manager.DataManager.loadConfig(DataManager.java:307)
	at com.conveyal.datatools.manager.DataManager.main(DataManager.java:78)
	at com.conveyal.datatools.DatatoolsTest.setUp(DatatoolsTest.java:25)
	at com.conveyal.datatools.LoadFeedTest.setUp(LoadFeedTest.java:28)
	at com.conveyal.datatools.editor.ProcessGtfsSnapshotMergeTest.setUp(ProcessGtfsSnapshotMergeTest.java:35)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
17/04/19 16:03:58.466 [main] INFO  com.conveyal.datatools.LoadFeedTest - ProcessGtfsSnapshotMergeTest setup
17/04/19 16:03:58.479 [main] WARN  c.c.datatools.manager.DataManager - Config property application.data.mapdb not found
...

add ignored stopID verification step to createTrip; fixes #43

Issue by landonreed
Monday Sep 25, 2017 at 17:43 GMT
Originally opened as catalogueglobal#44

See #43. This fixes an issue where the pattern stops were not being compared against the trip stoptime stop IDs when creating a new trip.

landonreed included the following code: https://github.com/catalogueglobal/datatools-server/pull/44/commits

'Fetch All' not fetching all updated feeds

Issue by landonreed
Friday Sep 15, 2017 at 12:33 GMT
Originally opened as catalogueglobal#35

@demory experienced this behavior when fetching feeds for the DC area. Only three out of 4 or 5 feeds were fetched after clicking 'Fetch All'.

fix(Editor): add check for permission to edit GTFS feed

Issue by landonreed
Thursday Apr 06, 2017 at 21:05 GMT
Originally opened as catalogueglobal#6

Perform a check before all api/editor/secure non-GET routes to ensure that the user has permission to edit the feed. This PR is in conjunction with catalogueglobal/datatools-ui#6

landonreed included the following code: https://github.com/catalogueglobal/datatools-server/pull/6/commits

Replace MapDB with MongoDB (application data) and relational database (gtfs data)

Issue by landonreed
Tuesday Sep 19, 2017 at 20:10 GMT
Originally opened as catalogueglobal#38

Do not merge, this PR is a WIP.

This PR uses updated gtfs-lib and gtfs-api versions to swap out the GTFSCache system for a relational database to store GTFS data.

It also swaps MapDB for MongoDB to store the main application data (lists of projects, feed sources, feed versions, detc.).

landonreed included the following code: https://github.com/catalogueglobal/datatools-server/pull/38/commits

Remove trip direction serializer/deserializer

Issue by landonreed
Wednesday Sep 20, 2017 at 19:53 GMT
Originally opened as catalogueglobal#42

According to #41, there was an issue deserializing/serializing pattern direction values from the A/B enum values to the 0/1 GTFS values. There is no need to deserialize into JSON with the 0/1 values, so this PR removes all of that activity.

This PR has a corresponding PR in catalogueglobal/datatools-ui#28

landonreed included the following code: https://github.com/catalogueglobal/datatools-server/pull/42/commits

Use threadpool

Issue by landonreed
Friday Jul 07, 2017 at 15:14 GMT
Originally opened as catalogueglobal#22

Prevent multiple threads from being started during heavy and light job execution.

landonreed included the following code: https://github.com/catalogueglobal/datatools-server/pull/22/commits

Cache problem

Issue by zbouziane
Friday Apr 28, 2017 at 12:57 GMT
Originally opened as catalogueglobal#13

...
12:53:33.747 [pool-1-thread-1] INFO  com.conveyal.gtfs.BaseGTFSCache - db files written.
12:53:52.178 [pool-1-thread-1] WARN  c.c.d.m.c.api.GtfsApiController - New version found for gtfs/STIF-20170428T093236Z-6e67d5ff-7a7a-48b4-a9b1-528a9f73c68c.db is null. Downloading from s3...
12:53:52.464 [pool-1-thread-1] INFO  com.conveyal.gtfs.BaseGTFSCache - Attempting to download cached GTFS MapDB.
12:53:52.525 [pool-1-thread-1] WARN  com.conveyal.gtfs.BaseGTFSCache - DB file for key gtfs/STIF-20170428T093236Z-6e67d5ff-7a7a-48b4-a9b1-528a9f73c68c-db does not exist on S3.
12:53:52.526 [pool-1-thread-1] INFO  com.conveyal.gtfs.BaseGTFSCache - Feed not found locally, downloading from S3.
12:53:52.949 [pool-1-thread-1] ERROR com.conveyal.gtfs.BaseGTFSCache - Could not download feed at s3://datacatalogue/gtfs/STIF-20170428T093236Z-6e67d5ff-7a7a-48b4-a9b1-528a9f73c68c-db.
12:53:52.949 [pool-1-thread-1] WARN  c.c.d.m.c.api.GtfsApiController - New version found for gtfs/STIF-20170428T093236Z-6e67d5ff-7a7a-48b4-a9b1-528a9f73c68c.db.p is null. Downloading from s3...
12:53:59.120 [pool-1-thread-1] INFO  com.conveyal.gtfs.BaseGTFSCache - Attempting to download cached GTFS MapDB.
12:53:59.160 [pool-1-thread-1] WARN  com.conveyal.gtfs.BaseGTFSCache - DB file for key gtfs/STIF-20170428T093236Z-6e67d5ff-7a7a-48b4-a9b1-528a9f73c68c-db-p does not exist on S3.
12:53:59.160 [pool-1-thread-1] INFO  com.conveyal.gtfs.BaseGTFSCache - Feed not found locally, downloading from S3.
12:53:59.167 [pool-1-thread-1] ERROR com.conveyal.gtfs.BaseGTFSCache - Could not download feed at s3://datacatalogue/gtfs/STIF-20170428T093236Z-6e67d5ff-7a7a-48b4-a9b1-528a9f73c68c-db-p.
12:53:59.168 [pool-1-thread-1] WARN  c.c.d.m.c.api.GtfsApiController - New version found for gtfs/STIF-20170428T093236Z-6e67d5ff-7a7a-48b4-a9b1-528a9f73c68c.zip is null. Downloading from s3...
...