opentripplanner / opentripplanner Goto Github PK

I think Vertex should have no setters. The underlying implementing class can be totally mutable, but I think it improves the safety of the routing code and reduces the barrier to implementing Vertex if the setters are removed.

On the [wiki:meetings/10-09-09 10/9 conference call] Bibiana said that she'd work on getting clearance to distribute and use their centerline file for OTP testing and development.

... because it lacks LION data from NYC. Need to provide LION downloading instructions and skip test if LION doesn't exist.

A large GTFS dataset like this will be very helpful in testing the planner.

Since most people haven't yet signed up for the TripPlaner-Dev mailing list, I was wondering how folks feel about using a Google Groups list instead. One advantage to this is greater familiarity and consistency, since most people have used it before and both Graphserver and FinePoints are already using Google Groups for their mailing lists.

Also, we should settle on a name before settling on a list (the current name, TripPlanner, was just a placeholder). Frank discovered at the workshop that OpenTripPlanner.org was available, and everyone seemed to like that name, so we've registered it (and I've pointed it at our Trac instance). See ticket #10. How do people like OpenTripPlanner as the official name for the project?

In short, are people OK with creating an opentripplanner-dev list on Google Groups and using that for the developer discussion?

Please post your thoughts as comments on this ticket, as I'd like to move forward with this as soon as possible so we can get down to the fun stuff :)

Thanks!

OTP presently requires Processing. Since this is useful only for
developers, it should be optional, with relevant features omitted if it is absent.

... but one-way traffic for cyclists on one-way streets except on start and end segments, and one-way traffic for cars on all one-way streets.

This turns out to be much faster than Dijkstra's algorithm -- but it requires that all vertices be of class spatialvertex. I think we can make this change pretty easily.

using block_id

The "target/" directory is the general output directory for Maven build artifacts (compiled packages, javadoc, jars, etc). Currently, the "target/" directory has been checked into svn. As result, svn will be looking to add any of the build artifacts in that directory to the repository, which we shouldn't be doing. I propose adding the "target/" directory to svn:ignore and remove it from the repository.

I will make this change on Oct. 16th unless I hear otherwise.

A particular stop can be visited multiple times in the same trip. For a real-world example, see "Childrens Hospital AcRd & 45th Ave NE" for the following trip:

http://onebusaway.org/where/standard/trip.action?id=13417665

I believe TripPattern currently won't support this correctly, as it has an internal Map<Stop,PatternStop> that will just overwrite a particular stop if it occurs multiple times.

Solution: Key off a Stop + index where index is the position of the stop in the original list of StopTimes. Will require modifications to TripPattern, PatternBoard, and GTFSPatternHopFactory.

Right now, opentripplanner-routing depends on slf4j-simple, which doesn't give us much control over logging directives. I propose the following:

1. Use slf4j-log4j as our actual slf4j plugin, since it has a lot more controls for picking log levels for different modules. Not actually 100% committed to log4j either, if someone wants to make the case for a different logger.

2. Only make the slf4j-log4j dependency a first-class (aka scope=compile) dependency in modules that are designed to be run directly (such as opentripplanner-api-webapp) and make it a test dependency (scope=test) in library modules (such as opentripplanner-routing), which make it easier for 3rd parties to override the logger when using the library, but still lets us run tests that produce logging output.

With multiple developers contributing code now, it'd be great to have an established Java code style and formatting guidelines.

I don't have strong feelings about what style to use, as long as it can be codified as an Eclipse Java code style file.

Does TOP already have a style guide? If not, we could use any of the following:

http://code.google.com/webtoolkit/makinggwtbetter.html#codestyle
http://maven.apache.org/developers/conventions/code.html

Comments by Friday, Oct 23 or I will just pick on of the above and check-in appropriate style files for Eclipse.

!OpenTransit? !TripPlanner? !OpenTripPlanner? Something else?

Summary: list all the artifacts needed by the end user of OTP; specify what interfaces the end user will then use to adapt the system for his data and his look & feel.

Scenario: Lowly DBA at transit agency X is tasked with getting OTP up and running (with his agency's data...eventually). Lowly has zero java/tomcat/mvn experience. How do you instruct Lowly what to download, where to install, what to run, where to load/edit, etc... ? (BTW, I've seen many people like Lowly, with zero java experience, install tomcat, deploy a .war file and load their own data into a system without much problem, so it's a manageable scenario ... see TimeTablePublisher.org [TTPUB]).

Current situation (November 2009): OTP code is broken into 3 modules. Packaging this up, we get one jar file (routing) and two war files (web service and Map UI).

Issues & Questions:

1. Having multiple .war files is great for separation, but our current .war naming scheme changes based on version number; thus, for the UI from A.0.1-SNAPSHOT.war to talk to the web service over in B.0.2-SNAPSHOT.war, we might need to change our .war naming to a constant name of say otp-ws.war and otp-map.war. Sound like a good thing?
   Voting: Frank (Author) +1
2. Where does Lowly hook into the code / data?
3. Map UI will need a properties file (or .js file) to personalize things like the initial extent (eg: currently hard coded to Portland).
4. WS API needs an API docs (and demo apps TODO???).
5. Data loader hooks are ???
6. What documentation to write?
7. TOPIC - Getting Java/Tomcat/OTP Running: This README for TTPUB has been about 90% successful, so it could be a starting point at least getting java/tomcat/.war files up & running: timetablepublisher.googlecode.com/files/Developer%20Notes%20v1.1.doc
   NOTE: one question not answered by this document is what are the hardware requirement for the system (an issue that only one person that I know of had...but this system is much larger / hungrier, so...)

• TOPIC - Customizing the map UI: TODO docs & examples ????
• TOPIC - Loading Data into OTP:
  • what data: GTFS/Centerline/Bike Paths/NED/etc... data will look like X and gets loaded into OTP by doing Y - TODO???
• TOPIC - Open Source License:
  • The (LGPL) license will be found (in every source file/here/?) TODO ????

We've had [http://groups.google.com/group/opentripplanner-dev/browse_thread/thread/1501f21d8d10f0bd/d3b801541fc82942?lnk=gst&q=license#d3b801541fc82942 multiple] [http://groups.google.com/group/opentripplanner-dev/browse_thread/thread/ef8312c409a4b39d/9538c7c0971ec168?lnk=gst&q=license#9538c7c0971ec168 discussions] [http://groups.google.com/group/opentripplanner-dev/browse_thread/thread/c65f6624a87fc9b/fdb2ab045b97ffd4?lnk=gst&q=license#fdb2ab045b97ffd4 about] [http://groups.google.com/group/opentripplanner-dev/browse_thread/thread/3604217b27fe4b19/17ad118ab67e39ae?lnk=gst&q=license#17ad118ab67e39ae licensing] and the deadline we agreed on for a decision is in a couple days (November 2).

I'm creating this ticket 1) so we can discuss it (briefly) on the call, and 2) have a clearer record of what's been discussed and what we ultimately decide.

My suggestion is still that we go with the LGPL since it seems to me the closest thing to a compromise between the values and preferences voiced in the aforementioned discussions.

Since Frank has opened discussion on broader module name refactoring, I figured I'd toss in my two cents:

I'd propose a more Maven-centric approach with what's called an aggregator or multi-module project. See http://maven.apache.org/pom.html#Aggregation for a broader description.

We'd have a root pom.xml file in trunk that defines global project settings (Java version, License, SVN location, Maven repository publishing settings) and then we've have sub-modules, with a sub-directory for each module and a pom.xml file in each sub-directory.

Each of these sub-modules would correspond to broad areas of functionality (routing, visualization, webapp, etc), so we could better separate functionality. I propose sticking to the the following naming scheme for modules artifactIds:

opentripplanner-${module_name}

The main thing is that by prefixing the artifactIds with "opentripplanner" (or maybe something shorter like "otp"?) we can avoid namespace collisions when importing the Maven modules as projects in IDEs like Eclipse. This seems to be a pretty common practice amongst Maven multi-module projects (see Apache Struts and Google GWT for two examples). Note that this is addition to the groupId, which would stay "org.opentripplanner".

I use a similar module layout for OneBusAway and it works well. I think a step like this is going to become necessary, especially as we starting adding different high-level components, such as webapps and other layers.

Allows for a more flexible implementation of Vector without the constraints/assumptions of the Vector class (synchronization, etc)

Since Portland is the first city that will actually be using OTP in production it'd be good to start testing and using its street data as soon as possible. Once we have their centerline file (#26), we need to make sure it loads correctly and we can link it to the transit network for handling multimodal trips.

Ultimately we'll want to have a flexible system for loading street networks for all cities that have their own centerline files. Having only really worked with NYC's street data, I don't have a good idea of how much variation there is between cities. This could either be straightforward or require a good bit of thought.

See http://wiki.github.com/bmander/graphserver/board-alight-crossing-transit-graphs

Some libraries to consider:

• [http://jgrapht.sourceforge.net/ JGraph] and JGraphT
  • of the two, JGraphT is the one we'd want to start with since it's aimed more directly at what we're doing; if we ended up needing visualizations, then we could also use JGraph)
  • seems to be pretty mature
  • latest release requires Java 1.6
  • LGPL-licensed
  • Five Points uses it, and David says he hasn't had any performance issues with graphs with 100k edges
• [http://jung.sourceforge.net/ JUNG]
  • relatively active (e.g., mailing list has a decent amount of activity)
  • last release April, 2009
  • BSD-licensed
• others?

Are they fast enough? How well do they handle very large graphs?

See some of the messages from this thread for more info: http://groups.google.com/group/opentripplanner-dev/browse_thread/thread/75a1fed716c18a68

All things being otherwise equal, there are definite advantages to using an existing library where possible. If it's mature and has a decent community around it, it's more likely to be more stable/have more bugs ironed out, it makes hacking on OTP more appealing to people already familiar with the library, improvements made to the lib we get for free, etc.

The State object clearly needs some absolute measure of time. I'd argue, however, that GregorianCalendar is a super heavy duty object that's not appropriate for this purpose.

The State object spends a lot of time being copied and cloned, and the GregorianCalendar.clone() operation is really slow. In a quick test I whipped up, incrementing and copying a GregorianCalendar object 10k times in a loop takes over 100ms, while incrementing and copying a long is < 1ms.

As an alternative, I think we should refactor State to use a simple long value to measure time. The general convention for measuring absolute time in Java is to use a long value measuring the number of UTC milliseconds since the Java/Unix epoch and I'd stick to that (see System.currentTimeMillis(), Date.getTime(), Calendar.getTimeInMillis()).

I acknowledge that there are complexities in dealing with Timezones, DST, leap-years, etc when we build our trip planner, but I don't think we will run into them at the State level. If you look at most of the edge operations working with State.time, most are simply increment operations of some number of seconds, which aren't affected by these issues.

We're presently generating transfers based on distance. We should instead generate them from transfers.txt in GTFS.

When a user enters their address and it is geocoded to a lat/long, we need to find the nearest vertex to that (or perhaps the nearest edge) to do geocoding.

Also for the end point.

On the call today I'd like to make sure any development work that people are doing is ticketed

The inconsistent formatting and presence of tab characters is making me batty. I propose we do a wholesale reformatting of all of our code using the Eclipse profile agreed on in #24. I know this sucks because it screws up the commit history, etc, but I still think it's worth doing (otherwise our code will continue to look weird and we're likely to accidentally check in formatting changes anyway as part of other commits).

Since we have several branches going we should probably hold off until they're merged to trunk.

We talked about this briefly on the call a few weeks ago. I propose we make this official and use EPSG 4326 lat/lon for everything (at least internally -- we can reproject as necessary in our output).

OneBusAway has existing libraries for reading/writing GTFS, as well as storing it in a database using Hibernate. I propose replacing the existing gtfs functionality that Brandon wrote with the OneBusAway libraries.

A couple reasons why:

1. This library has been in production use by OneBusAway for a while and it's well-tested.
2. The OneBusAway GTFS libraries have been carefully designed to deal with multiple agencies in the same data store. By multiple agencies, I mean handling multiple GTFS feeds in the same graph/database/etc. The GTFS spec makes no guarantees that stop, trip, and other entity ids will be unique across GTFS feeds, and I believe the current OTP GTFS codebase will start running into issues when we attempt to interleave feeds from geographically nearby transit agencies into one graph.

Documentation about OneBusAway's GTFS libraries:

http://code.google.com/p/onebusaway/wiki/ModuleGtfsExamples
http://code.google.com/p/onebusaway/wiki/ModuleGtfsHibernateExamples

Now a couple reasons why not:

1. Brandon, I know you looked at my library at some point and decided not to use it. I'm interested in hearing your reasons.
2. Currently, the onebusaway-gtfs-hibernate module has a dependency on the Spring IOC framework. I can remove that if it's a big deal.

I'm happy to do the work to transition the existing codebase, but since it's a non-trivial change, I'm definitely looking for feedback / votes.

Presently the graph is loaded into memory from base data sources at runtime. This is a fairly expensive process. William Lachance in libroutez and Brandon in graphserver both compile to a persisted graph. Such an arrangement speeds up developer time, because:

(a) each test run no longer requires a re-build of the graph
(b) the compiled and persisted graph can be inspected cheaply
(c) the graph can be compiled in O(1) memory

Wyatt [http://groups.google.com/group/opentripplanner-dev/browse_thread/thread/122024694af389ef commented on the mailing list] that we should have a clearer outline for meetings to help focus them. Specifically:

1. We should start with a check-in (i.e., everyone says what they've been working on or simply "nothing to report").
2. If a check-in isn't going to be brief (i.e., you want to discuss some technical challenge/decision), it should on the agenda separately
3. Items (particularly those that require a response beyond yes/no) should be on the agenda ahead of time

I propose we adopt the above three guidelines. I want to balance having a useful structure to keep meetings productive with keeping the overhead low. We can achieve the latter by automating things and making the process self-documenting. How? Here's one idea:

If you want to add something to the agenda for a meeting, create a ticket (if there isn't one already; most things we'll be discussing will hopefully already be ticketed) and add a keyword, e.g., {{{agenda-10-23-09}}}. This has the following advantages:

1. It's pretty darn easy to do
2. We can automatically generate the agenda using a TracQuery embedded in the wiki
3. It reduces redundancy/the chance that information will be in multiple places and therefore become out of date
4. It's self-documenting; the agenda will automatically be updated

As a trial, I've set this up on [wiki:meetings/10-23-09 this week's meeting page].

I've started in on this.

http://opentripplanner.org/browser/branches/ws-ui

Done so far:

1. Jersey ReST service
2. Minimally running API (static place holder content in a lot of places)

Plan to do:

1. Add an web application that then uses the API.
2. Build out the minimal API (and probably use a lot of static data in there initially) to get APP and API talking.
3. Start tying the API to the real trip planner back end, and start rendering results in the UI.
4. Look to changing the API to reflect what others have specified above (eg: first pass of the API will look a lot like the TriMet api ... again above).

OTP needs to be able to load National Elevation Dataset and use it to assign elevation data to edges representing streets. This is useful for calculating optimal walking and especially bicycling paths.

The Open Street Map importer from Graphserver should be ported to OpenTripPlanner.

One more thing to point out about using Java is you can use GeoTools to load data from PostGIS. Loading from PostGIS I believe is one of the requirements for this project. Of course in C you can use OGR just as well. But Java will make it possible to do nice integrations with GeoServer. GeoServer already provides a GUI framework to configure datastores. So with a bit of work you could have people configure their data, with Oracle or ArcSDE or PostGIS or whatever, in GeoServer, and then take just one additional step to designate which is roads, transit, bike routes. And we'd be able to distribute it as an easy plugin to GeoServer. And OpenGeo will be able to lend support to package it up for better GeoServer integration.

From [http://groups.google.com/group/opentripplanner-dev/browse_thread/thread/482dcd872fc02f82 this thread]:

{{{
We should formally document our decisions making process on the wiki so
newcomers can easily get up to speed. We've been informally using +1/-1, but
haven't firmly defined anything. I propose we continue with the following
additions/clarifications:

+1 - I support this
+0 - I don't have a strong opinion, but am not opposed
-0 - I'm against this, but don't have a good alternative / I'm not willing to
do the work on the alternative / I won't block
-1 - Blocking no, but generally should require people to propose something else
that they would put the time in to.

Chris recommended this to me since he's used it in a few other projects with
success (it's borrowed in from Apache, which has a more granular version:
http://www.apache.org/foundation/voting.html) and helps differentiate between a
weak vote against something and a strong blocking vote. (I'm cool with dropping
the +0, since I don't think it adds much.)

Assuming no blocking objections, I'll document this on the wiki Monday, 10/26.
}}}

The standard location for resources used in unit testing in Maven is "src/test/resources". I propose moving the "caltrain_gtfs.zip" and "google_transit.zip" files there an updating the unit tests as needed.

I also propose removing "bart-archiver-..." since it doesn't seem to be used in any of the unit tests. Correct me if I'm wrong?

I think this would clean up the source tree and make it clear the purpose of these files in the tree.

Feedback deadline: Oct. 16

For Vertex, make it an interface with the following methods:

Iterable getOutgoingEdges();
Iterable getIncomingEdges();
String getLabel();

(I might also suggest getLabel() => getId() to indicate that it should ideally be unique?)

Combine Edge and Walkable into one interface:

Vertex getFromVertex();
Vertext getToVertex();
WalkResult walk( State s0, WalkOptions wo );
WalkResult walkBack( State s0, WalkOptions wo );

By moving from the current public field approach on concrete Vertex and Edge objects to an interface approach, we can hide the implementation of where those vertices and edges are coming from. This means that we don't necessarily have to have the entire graph loaded into memory, or at least not in the full Vertex+Edge form. Instead, we might have some optimized underlying data structure from which the Vertex and Edge objects are generated as needed.

By combining Walkable and Edge, we remove a layer of separation that I don't think is really buying us anything at this point.

Novalis has already started working on this.

Don't close until initial architecture and interfaces between the components are defined and we know the narrative fits the bill.

Going to close this. The UI is in OK shape, and going to get a whole lot better after Nick checks in the TOPP work done for TransitTech.

(That said, I know there are a bunch of things I want to do to this UI eventually).

I think our current use of the term "walk" in a number of graph operations is a little loaded. Specifically, the Walkable interface, along with the WalkOptions and WalkResult classes, always suggested to me the actual act of walking by foot, when in fact they refer to general graph traversal. I think this might lead to confusion in the future.

To promote the general understandability of the code, I'd propose the following renames:

1. Walkable => Traversable
   1a) Walkable.walk => Traversable.traverse
   1b) Walkable.walkBack => Traversable.traverseBack
2. !WalkOptions => !TraverseOptions
3. !WalkResult => !TraverseResult

Generate a topographic map of the trip. See attached graphic or use Bike-only option on FivePoints for an example.

I propose bumping the minimum Java version for JAGS from Java 1.5 to 1.6 for the following reasons:

1. Java 1.5 officially went into End-of-Life on October 8th (http://java.sun.com/j2se/1.5/). Sun won't be releasing critical fixes on the 1.5 branch anymore.

2. Many Java libraries have started appearing compiled for 1.6 by default, which is not compatible with 1.5. Switching to 1.6 for our own code means we can still use old 1.5 libraries AND new 1.6 libraries.

The switch involves bumping 1.5 to 1.6 in pom.xml.

I am trying to load the Brooklyn NYCT buses into OTP, and I run out of memory. The GTFS is: http://data.topplabs.org/data/mta_nyct_bus/bus-B.zip

This is probably under 20% of the total NYC area transit data, as it is just buses from the larger of the two bus companies for largest of the five boroughs, and does not include any subways or commuter rail, let alone non-MTA transit agencies.

<error message="Java heap space" type="java.lang.OutOfMemoryError">java.lang.OutOfMemoryError: Java heap space
    at org.opentripplanner.jags.core.Graph.addEdge(Graph.java:50)
    at org.opentripplanner.jags.core.Graph.addEdge(Graph.java:60)
    at org.opentripplanner.jags.edgetype.loader.GTFSHopLoader.load(GTFSHopLoader.java:36)
    at org.opentripplanner.jags.edgetype.loader.GTFSHopLoader.load(GTFSHopLoader.java:41)
    at org.opentripplanner.jags.edgetype.loader.GTFSHopLoader.load(GTFSHopLoader.java:45)
    at org.opentripplanner.jags.test.TestNYC.testBasic(TestNYC.java:25)

I can fix this by setting my heap size to 1.5GB using the following method: http://www.mail-archive.com/[email protected]/msg50651.html

But I don't have 15GB of RAM to load the full data set -- or even 5GB to load all the NYCT buses.