Code Monkey home page Code Monkey logo

expfactory-surveys's Introduction

The Experiment Factory

DOI DOI Gitter chat

expfactory/static/img/expfactoryticketyellow.png

See our documentation for getting started. If you are new to containers, read our background or paper first. If you want a more guided entry, see the detailed start

The Experiment Factory is software to create a reproducible container that you can easily customize to deploy a set of web-based experiments.

Citation

If the Experiment Factory is useful to you, please cite the paper to support the software and open source development.

Sochat, (2018). The Experiment Factory: Reproducible Experiment Containers. Journal of Open Source Software, 3(22), 521, https://doi.org/10.21105/joss.00521

Contributing

We have many ways to contribute, and will briefly provide resources here to get you started.

How to Contribute

If you are a developer interested in working on the Experiment Factory software you should read out contributing guidelines for details. For contributing containers and experiments, see our user documentation. If you have any questions, please don't hesitate to ask a question. You'll need to lint your code using black:

$ pip install black
$ black expfactory --exclude template.py

Code of Conduct

It's important to treat one another with respect, and maintain a fun and respectful environment for the open source community. Toward this aim, we ask that you review our code of conduct

Background

It's predecessor at Expfactory.org was never able to open up to the public, and this went against the original goal of the software. Further, the badly needed functionality to serve a local battery was poorly met with expfactory-python as time progressed and dependencies changes.

This version is agnostic to the underlying driver of the experiments, and provides reproducible, instantly deployable "container" experiments. What does that mean?

  • You obtain (or build) one container, a battery of experiments.
  • You (optionally) customize it
    • custom variables (e.g., a study identifier) and configurations go into the build recipe
    • you can choose to use your own database (default output is flat files)
    • other options are available at runtime
  • The container can be easily shared.
  • You run the container, optionally specifying a subset and ordering, and collect your results

If you build on Docker Hub anyone else can then pull and use your exact container to collect their own results. It is exact down to the file hash. Note that bases for expfactory were initially provided on Docker Hub and have moved to Quay.io. Dockerfiles in the repository that use the expfactory-builder are also updated. If you need a previous version, please see the tags on the original Docker Hub.

Experiment Library

The experiments themselves are now maintained under expfactory-experiments, official submissions to be found by expfactory can be added to the library (under development) to be tested that they meet minimum requirements.

expfactory-surveys's People

Contributors

ianeisenberg avatar rwblair avatar vsoch avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

expfactory-surveys's Issues

More data needs to be added to the data output by surveys

While the surveys are currently much more beautiful, they don't save nearly as much data as the previous ones did. Right now they record the scored choice that the person made, and that's it. (For @zenkavi) Reverse scoring is taken care of by defining "option_values" in a survey's .tsv see here

Minimum data we need:
-The actual choice the person made. So if they chose the first option, we need to know that.
-Response range (scored out of what?). This can just be the whole option field to allow for non-likert like scales (option text may be [3,4,17] or something else wonky)
-The question text

Other things that @zenkavi recorded that was nice:
-Reaction time per page of the survey
-stim_response text
-times viewed (I presume that's the number of times someone visits a page if they use the back button)

@zenkavi, I don't have the bandwidth to deal with this, at least not alone. Could you own this (and we need the surveys fixed ASAP). I'm happy to meet up so we can work on it together. @vsoch, could you point us in the right direction? Where is the dictionary returned here defined.

Interested in contributing surveys

Hi. I'd like to contribute surveys but I am a bit confused.

  1. Where am I supposed to contribute surveys?

https://github.com/expfactory/expfactory-experiments

or https://github.com/expfactory-experiments

or https://github.com/expfactory/expfactory-surveys

or anywhere else?

If the first one, how do I request a pull?

  1. It seems there are two different formats for config.json. Which one am I supposed to follow?

https://github.com/expfactory/expfactory-surveys/blob/master/bis11_survey/config.json

or https://expfactory.github.io/contribute

If the second one, how do I write url before my pull request is accepted?

  1. Is there anything else to write apart from config.json and survey.tsv for contribution?

Thanks in advance!

surveys require --folder to specify a full path name

This works

expfactory --run --folder /full/path/to/expfactory-surveys --survey state_mindfulness_survey

but this,

expfactory --run --folder expfactory-surveys --survey state_mindfulness_survey

returns

No battery folder specified. Will pull latest battery from expfactory-battery repo
Currently only support running one survey, will run first in list.
Deploying survey state_mindfulness_survey
Found 30 valid surveys
Found 10 valid games
Invalid survey state_mindfulness_survey not found in surveys repo!
Please specify list of comma separated experiments with --experiments

This is not clear in the docs, so I'd suggest adding an example, or supporting relative paths with --folder.

three surveys missing responses

the number of responses / values are misaligned for three questions in a survey:

image

easiest way to find would be programatically

add time taken to complete surveys

This would be easy to do - add a hidden field to the form for a start time and end time, and then javascript to call the function getTime to populate the form when the users starts and submits.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.