rjaltman / dogu Goto Github PK

By the spec:

Project Create
Any member can create a new project by entering:

1. Name
2. Description
3. Target University (optional -- will cause project to only be displayed to this university)
4. Descriptor tags

If the member creating the project is a representative, they can choose which of their organizations to attach the project to.

When you add tags on the project create page, it looks like they show up on the UI, but they don't appear in the database.

Tags and universities are two specialized instances of text entry where there is a large cost to a typo but there are too many options for a dropdown menu to be reasonable. Writing "University of IIInois" in a text box by accident is not hard to do but carries a large cost, since now search doesn't work properly for that project (note that there are no L's in that string). Similarly, having a tag car and a tag cars also makes search less effective. But of course, having a dropdown menu for either of these both disallows new ones from being instantiated and would be a huge pain to use. Fortunately, someone came up with a solution in the form of what was called, at least during the first Obama administration, a typeahead. Here is an example. It allows you to search through a lot of options and then pick the one you want, which is then autofilled in the the text box for you. It works similarly to the address bar in a browser. This lets you have a huge number of options without having to waste time and make mistakes typing out whole long strings. It also helps user experience by giving examples of what should go in a box. This particular library is a relic from an era when people used jQuery in the presence of other options, but there must be some similar one that knows about Soundcloud rappers and Tide Pods. I think we should implement it for universities and tags, and perhaps some other things if we see fit.

By the spec:

Project Tags Edit/Delete
Users should be able to edit and delete existing project tags, along with adding new tags.

A rudimentary implementation of the goal home page should tie the existing functionality together by giving a logged in user the ability to access the project search and project add functionalities.

If you open the postgres client and say select * from pg_stat_activity, you'll see that the queries never really disconnect from the server, which causes problems in the long run. We should probably figure out how to fix this. I thought running the cursors in with statements would work, but it did not. Maybe it would work better if we ran cursor.close() before returning or something, but I dunno. Or we could just turn on autocommit. But hopefully we can fix this eventually because it's only going to get harder to fix from here.

See title

This should be simple enough, but we can't forget it.

I suggest we go through last semester's 411 projects and add them to the database. In order to demonstrate advanced features, we will also need to add a class, and create a number of accounts in that class with some project preferences for each of them.

Here's what we'll need: (these can be done in a migration)

• Clean up the current data (the main thing that needs to stay is the University of Illinois)
• 25 Projects from CS 411 Fall 2018
• A few corresponding tags for each project
• A CS 411 Course listing
• An instructor account for Abdu

To be created manually, after the above data:

• 10 student accounts (us four, and 6 friends that I've recruited) self-assigned to CS 411
• Project preferences for those accounts

Right now, the react Component used to register users is just part of the Login component. This is fine for right now, but the spec explains that the registration part has to have a bunch of complicated stuff in it, mostly the fact that the necessary information is a function of what kind of account you're making. Also, the astute reader will notice that some of the columns specified are not actually in the database yet, so someone will have to make those.

Another important issue that I don't think has been properly addressed anywhere is how people will pick their university. We could conceivably just have a textbox, but that's a recipe for disaster and would earn us untold but well-earned ignominy in the web development community (perhaps the two are related...). Instead, I propose we have something like this (http://twitter.github.io/typeahead.js/). This would alleviate the need for users to type the long name of their university, and it would also prevent typos, which would be very frustrating. The only downside is that we would need to find an exhaustive list of all the universities that we might need, which might be nontrivial.

By the spec:

Individual Project Page
This page will display a project and all its information. If this project is open and visited by a student without a project/group, the student will have the ability to add the project to their preference list.

The basic version needed for the demo should display the basic information: the project title, description, organization, and tags. Should also lead to the edit/delete functionality for the project's tags.

In the future, we should validate some core functionality based on a series of tests. Since we are using JavaScript for some of this project, we can accomplish parts of that with something like IstanbulJS, but we should probably have some different test run tools for the database itself. I might also recommend Coveralls, which will eventually get us looking into code coverage here.

Allow students to add themselves to a class. This would need a page listing all the classes, where students could select the one they need. While this could have a search or class entry keys or something fancy, we can also just make it work like PrairieLearn's.

As title. This can be viewed by going into Change Preferences on a project page.

This should be a really simple fix, just formalizing the issue so we eventually get to it.

If you send two requests too fast, one will still be "in progress" a little and the second will fail immediately. It has something to do with threading and stuff and only happens on heroku.

I think if we wrap all our database functions in with conn: or just say conn.commit() after we're done with the database, it will be fixed.

I see this as essentially being an implementation of the recommendation query created for demo 1. Ideally, it would show up on the user's home page, and could use the same card design currently in the search.

One thing to think about would be what this should show if a user has yet to preference any projects.

I can see two main ways to do this:

• Allow (student) users to add projects to their preferences from the project's page, ranking them sequentially when added. Students could then view and adjust their rankings on some sort of separate page.
• Everything is handled by some sort of preferences page, where students can add a preference by selecting a course somehow, and adjust the ranking on that same page.

I think version one makes a lot more sense, as it would be able to use our existing project search and view.

For an actually functional site, this would likely be needed.

This would involve adding a creator/owner field to each project, and only showing the edit and delete to that account.

I think this is kind of complicated, so we probably shouldn't do it yet.

I dunno. It should rollback the whole migration and fail. I should check on this.

Right now, testing (and by extension, using) the app is a bit of a clusterfuck. It works, basically, and it looks nice, but every time I test creating a project, I have to refresh the page and login again to go to the search page and test. That's obviously not sustainable.

There are two options for fixing this, the classic twain of doing it ourselves and using a library. To do it ourselves, we'd basically have to have a big switch on some pages that just checks what the URL is and has a mapping to what should be shown. We'd set a listener on the back button using the History API and we'd have to push new stuff onto the History stack whenever we want to do anything. Getting everything to work together (if, for example, we have any nesting going on) could be tricky.

There's also React Router (https://reacttraining.com/react-router/web/guides/quick-start), which is common enough that I've heard of it and seems old enough to have had all the kinks worked out. You can look at the link to see how it works, but the idea is that you just declare the routes similar to an old-school backend-based solution and then it takes care of a lot for you. It doesn't hide things too aggressively, but it does simplify the occasionally complicated History API.

I usually hate dependencies, but I think in this specific situation that we should use React Router. If anyone has any strong opinions on which one of these we should do, or if we should do anything at all, speak now or forever hold your peace. Otherwise, I am going to start implementing this at the end of this week.

Currently, when a project is created, nothing happens. It would be good to switch to that project's individual page once it has been inserted.

The project search currently only displays a plain text list of projects. It would be really great if you could click on one and can view its individual page.

This could be related to styling the project search, if you want.

On smaller resolutions that are present here, there are some pages where scrolling will remove the header. This should be a relatively smaller fix, just needing to update the position and z-index used here.

Add some sample data for the demo.

Data will include:

1. UIUC as a University
2. A selection of projects
3. A selection of organizations
4. A demo account to use

Will also:

• Clean up test data that has been inserted so far

This shouldn't really be that hard, but it will require me to mess with heroku a whole bunch, and heroku builds take forever now, so it might be a pain. Nonetheless, we can't just spam python files in the base folder, and we can't just have a giant file the holds all the backend code, so this is what we're going to have to do.

Necessary changes include:

• Changing enum possible types to enums
• Changing serial foreign keys to ints, to allow NULLs in those fields

Right now we can edit/delete tags on existing projects, but projects should be allowed to have tags when created. This could likely use a similar interface as to what's currently on the edit/delete tags.

As of now, there is no way to log out of the system, outside of re-starting the session all over again. While this is another thing that we should keep in terms of tracking sessions (with cookies), we should at least have a button in the header that supports sign out.

A fair amount of our current designs are centered around having some sort of logo, or associated information for a University, profile, or project itself. Right now, we are using temporary resources to handle all of this, but we should look into image hosting (e.g. via AWS), image linking, or some sort of module that will allow for image management and linking. This may require a modification to the database, using a new attribute that holds a string to a URI resource.

It's occurred to me that we have to have password-based authentication, but we don't have an SSL certificate on heroku. That means that if we do this the usual way, everyone's password will be sent in cleartext over the wire, which is obviously suboptimal.

My solution is to arrange a system whereby no authentication information sent over the wire will be useful for more than 10 minutes. The idea is that we send hashes over the wire, but they are salted using one-time use random numbers, meaning that as soon as they are read by the server, they can never be used again.

Here's a more detailed example.
A client C wants to log in. C tells the server S that C wants to log in using username U. The server generates a random number N and sends N, along with the salt of U's password, back to C. Recall that the user has just typed their password, so C has the password P and U. Now it also has the password's salt on the server and a random number. C hashes P with the salt, and then hashes the result of that with N, yielding a new hash H that will never be seen again, as N will never be generated again. C sends H to S, which then takes the salted password hash from the database and hashes it again with N just like C did before. If the server's result matches the result that C sent, S knows that C has the real password, but the password was never explicitly shared. S sends a session key K back to C, stores K in the database, and deletes N from the database. K will expire in a short time (say 10 minutes), after which time S will tell C to authenticate again. If there is an attacker reading the wire, all he has is H, which is now worthless, since it was hashed with N which will never be used again, and K, which will only be useful for the next 10 minutes. The attacker has the salt of U's password, which isn't that valuable, and U itself, which he could get easily anyway.

"But wait," you're probably asking, "won't this make the site a huge pain to code?" No. At least, hopefully not. I have a plan for that too.

On the JavaScript side, I will have an object (not a React Component, mind you) called an Authentication. An Authentication will contain the salted hashed password, a session key, and the username of the current user. It will have methods for POST requests and GET requests that will abstract away the need to worry about any authentication problems. If will automatically include the session key in the payload of the requests if required, and if the key expires, it will automatically re-authenticate and retry the request. In theory, you'll have a Login React Component that will accept login information and the return an Authentication object to the App Component through a callback (hopefully putting it in App.state). This means that the check for whether someone is logged in on the client side will just be App.state.auth !== null.

On the server side, you can just use a decorator (eg, @session_required) to turn route functions into authenticated route functions. All that decorator will do is look at the Flask request object and find the session key in the payload. If the session key is still good, it will set a global variable user with the primary key of the User object that is calling the API, otherwise it will return an error saying that the session does not exist. This will happen often as sessions expire after a short period of time.

Note also that the User object will be global to match the request object in Flask. I assume if they're doing it, it must be a pretty good idea. They have a method for dealing with multithreaded web servers, which I will steal to implement this.

Having said all that, this is a request for comments. I think this will work, and it'll be cool, and it will be secure, but it will probably also be somewhat complicated. I think I've come up with abstractions that will make it simple to use, but this remains to be seen in practice. Any thoughts?

In e51828f (and a bunch of others) we slightly broke things so we wouldn't have to worry about them. We should probably fix those things long-term before we forget.

We have decided that one of our advanced functionalities will be to match students into project groups, based on their interest in certain project ideas. There's still a lot to think about here:

• If too many students put a project as their first choice, how is it decided who gets that project?
• How do we make sure every group is of sufficient size? (if only one student preferences a project, even if that project is their first choice, it should be skipped over, etc)
• Do we ever match a student with a course they didn't rank? (This would likely be necessary in any class) How is that decided?
• When do we drop a consistently ranked project, because a different grouping of students would simply be more ideal numbers wise?

Basically, this will need some sort of algorithm, far beyond just SQL queries. It's basically the stable marriage problem, but with an uneven mapping. Maybe closer to the assignment problem. Either way, this will be a lot of optimization. I'm going to do some thinking.

From the spec:

Project Search
A section where any member can search projects based on keyword, title, organization, or class. Whenever a search is being performed, the searcher also must satisfy the project statuses they are searching for: open/unassigned (unstarted or paused), in-progress, completed. This defaults to open.

Keywords, nota bene, are in a separate table, which I missed at first, and the project.description column should probably also be included. Since we'll be searching user-generated text for instances of other user-generated text, we'll probably want to use Postgres's super full-featured full-text search. It's somewhat complicated (so much so, in fact, that we might be able to pass it off as an advanced feature), but it's incredibly powerful, so I think it might be worth it.

At the time of this writing, the status row of projects, the referent of the last part of the quoted paragraph, is just an integer in the database. This should probably be changed to an enum at some point (#16), but a question remains as to when. I wouldn't go so far as to say that this ticket should be blocked on that change, but the current arrangement certainly adds complexity to a feature that is already plenty complex.

Another question (which I pose to all collaborators) is whether the design work on this feature should be done as part of this ticket or separately. If this does contain the visual part, that will simplify things somewhat, but it will also be a much larger task, one which might be difficult or impossible for a single person to do adequately.

That was a substantial oversight...

Since the group matching requires there to be a maximum amount of preference rankings, (maxrankings field in the course table) there should eventually be some way to prevent a student from going over that amount of project preferences.