stanfordhci / bang Goto Github PK

db.users is getting set to give bonuses but the bonus is not being paid, and db.users is not getting reset to 0.

Everything we change to run should be behind one meta bool.

vue.js or similar, letting us insert text from the server more easily.

If someone leaves the task during the pretask, it kills the task for everyone.

err: cannot read property results of undefined

MUST FIX BEFORE DEPLOY

Make an ad group
Kill an ad group
Check up on an ad group

Deal with situations where someone leaves or the system breaks and we still pay people.

The tricky thing is we need the task to be submitted to be able to pay them, so perhaps we autosubmit in some situations?

Also we can work with continuous payment, e.g. small upfront amount, and then pay a consistent hourly wage based on time working. In this case we could have a different payment level for waiting and for working.

Right now there's a few toggles, e.g., const midSurveyToggle = false, intended to enable switching behavior depending on what we are trying to do with a particular run of the tool.

@maikaisogawa could you update the midSurvey too support this, and others in @StanfordHCI/scaled-humanity, can you ensure any other modifications you are adding have such controls. Of course if there seem to be other important controls for a particular modification, it would be good if we could collect those in one place so we can manage experiments easily. If we are doing AB testing, it will be helpful for us to be able to modify how a particular user sees the interface based on these toggles.

Scale deployment: this scale should be deployed once (for the team) after every group task (same as mid-survey)
Scale:
Psychological Safety Scale (Edmondson 1999)
Traditional 7-point Likert scale: strongly disagree, disagree, neutral, agree, strongly agree)
Textfile: https://github.com/StanfordHCI/bang/blob/master/txt/psychologicalSafety.txt

This might be a little tricky and is low priority.

@phoebek you might know a good way to do this on the vue side. I'm happy to chat about dealing with getting it set up from the server.

We need an endpoint where if we have to kill it, we can navigate to that URL and it will:

1. Push everyone to a page that explains that the experiment will stop and how they will be compensated.
2. Upon submission, we pay each user and bonus them
3. Our interface reports individual payments and total (either in console or at the endpoint)
4. Kill the node app

The line below is called within the reposting function but i is never initialized. Which I think might have been related to bulling this out of the prior loop.

@maikaisogawa you might know the best way to fix this.

Expose things like runBang and scheduleBang (for setting up future tasks)

I think making it independent from the mturk integration would be ideal.

Happy to chat about this if there's questions or ideas.

Vision:

• Blank page after the task is terminated with the something like this (I'll wordsmith more later):
  "Thank you for your time! Unfortunately, our group task requires a specific number of users to run, so once a user leaves, our task cannot proceed. To receive payment for this task, please provide suggestions for how to design our study better to prevent this issue: "
• Open feedback form (like Alex's feedback button)
• Hit code after submission

So that we don't need to set the pricing amount every time we deploy.

Scale deployment
This scale should be deployed equal to the # of the team members in a group (i.e. after group 1, person A goes through a series of surveys, that each specify the name of the person on their team they are rating)
E.g.
Team with 4 members: Member A sees a survey asking him/her to rate team member 1, then team member 2 and so on….

Group Performance Scale
Scale measures
1 = far below average, 2 = below average, 3 = slightly below average, 4 = average, 5 = slightly above average, 6= above average, 7 = far above average
Textfile: https://github.com/StanfordHCI/bang/blob/master/txt/groupPerformance.txt

How to reproduce...
(e.g. when Drew typed in CLASSYDUCK, it auto-filled and left what he’s already typed (i.e. classyduckDUCK))

• add any relevant build variables into .env
• check that .env is work
• make new keys and put them in .env
• make sure old keys don't work and are removed from code

Force near complete to be completed and spell checked.

An issue from usability testing:

• Messages/names from previous round can roll into another round (see attached screenshot and ask if you have any questions)
  

-for all messages in room x/round y
-for all messages by user z (in rm x/rd y)

Kick users out after X minutes in the pretask waiting room

Set up a notification that lets the user know there is a chat room message when their screen is out of focus (e.g. when they look at the Kickstarter page during brainstorm).

And update credentials once that is known to work.

If pages are open from prior runs, disconnecting them after running again breaks. TODO noted around line 190 in server.

@maikaisogawa if its fine, feel free to just close. I was just alarmed to see a space there.

• Let people come and go without causing problems to running experiments
• Deal with overflow by running more experiments when possible (this means being able to run more than one experiment at once, which means having an experiment specific control flow.

Let users know what's going on so they don't think its broken.

1. If a user drops out during the pre survey we can't recover, as in we've already kicked the users over to be running and we can't deal with the situation where one of those users leaves. Theoretically we should be able to delay consequences of that departure until the teams are actually running.
2. If more users are in the waiting room than needed, and users start clicking in, there's still a potential point where we go beyond the desired number of users. e.g. we're expecting 4, we get 5 who accept, all of them can click to get started. I think we should basically not let that happen, and instead disable the button if the group is full, and then reenable it if we get a drop out before running.

Both of these are not critical for running, but would make us more robust.

Scale Ranking
1 = far below average, 2 = below average, 3 = slightly below average, 4 = average, 5 = slightly above average, 6= above average, 7 = far above average
Scale deployment:
This scale should be deployed equal to the # of the team members in a group (i.e. after task 1, person A goes through a series of surveys each specifying the name of the person on their team they are rating)
E.g.
Team with 4 members in a team: after each task member A sees a survey asking him/her to rate team member 1, then team member 2 and so on….
Ping me with any questions
Textfile with questions https://github.com/StanfordHCI/bang/blob/master/txt/statusCharacteristics.txt

Vision:

• Button in the top right of the task with something like "End task";
• If a user clicks on this button, show a blank page with the following message something like this (I'll wordsmith more later):
  "Are you sure you want to end the task? This is a group task that requires a specific number of users; if you leave now, you won't receive payment and other users will be unable to proceed with the task."
• Answers: 1. "Yes, I'd like to terminate the entire task." 2. "No, I'd like to finish the task and receive my bonus."
• If a user picks option 1, then deploy message: "We're sorry you decided to end the task early. If you'd like to receive a small bonus, please briefly explain why you choose to leave this task," and open feedback form (like Alex's feedback form)
• Hit code for bonus

@alliemb note what we need for each thing.

• Make sure all data in DB is accessible as JSON
• Build a script to convert JSON to CSV so we can use easily in R.

There's a few ways to do this. Chat with me (@markwhiting) if you're having a hard time fining a good approach.

• ensuring distance between users in a round
• ensure naming distance between users in different rounds

@maikaisogawa it sounded like you knew a fix for this, could you pus that if you do?

Set up hosted R notebook so we can process online, ideally do this on the main EC2, or a second one that links in so we don't need to do any extra steps to find remote data.

If user is wildCow, and I type cow, I don't see a UI to select wildCow, but I do get the autocomplete in most cases.

& change file directories in server.js

Approximately 5 min survey

1. Delay messages
2. Add info about users in the room
3. Make chat input area more visible as an input