How do we bring the test-integration into the test-automation project?

Concerns are:

• Keeping the repo relatively clean and hopefully free of having to include all of tutor-js
• Tangentially related -- how do we decrease the amount of processing/building we would run on Sauce Labs

We are keeping in mind that test-automation needs to be easily configurable to test both remotely (for QA) and locally (for development).

Current ideas

• use pip
• use npm
• Git subtree, submodules, or bash
• Build and host helpers separately
• Include all of tutor-js

use pip

Why

test-automation is currently mostly a Python repo

Why not

However, that will not be the case in the future, as tests will be written in some form of JS. The planis for current Python code to be pulled out to a package and the Python code will be used as command-line scripts.

use npm

Why?

Tests will be in JS so loading with npm will make it very easy to load in the necessary helpers without additional work. Also, npm will be part of the workflow anyway.

Why not?

Installing the whole repo into test-automation is a bit excessive. This would add time for Sauce Labs when installing and building tests. However, would handle this nicely.

If module size remains a concern, finding ways to install the specific folder (pulling out test-integration into a separate repo) could work, but would likely add additional overhead for maintenance.

Git subtree, submodules, or bash

Why?

Pulls in just the parts needed

Why not?

Involves writing an extra script that is likely to be complicated and require maintenance.

Build and host helpers separately

Why?

Less for test-automation to pull in, there will be no extra files locally.

Why not?

Maintenance overhead, need to do some extra steps to load in remote JS, uptime of JS file becomes crucial, adds debugging difficulties

Include all of tutor-js

Why?

Regardless of how, you can avoid the size problem and send only the built and minified JS to Sauce Labs.

Why not?

Extraneous files when working locally

Suggestions?

Any other ideas? We are not looking for a final decision, just ideas and potential pros/cons before we start trying some things.

/cc @openstax/tutor-fe @openstax/test

Description

Allow teachers to back out of a Willo integration manually if they select the wrong course, and decide if we can/should add controls to a teacher's UI that allow them to undo an integration.

Acceptance

• Admin can unpair a Willo integrated course from a Tutor course
• CS can walk instructor through what will happen if we back him/her out of an integration
• Decision on whether to add additional admin controls on the teacher interface for undoing LMS integration

Checklist for Done

1A-DEFINE

• Description is complete
• High-level natural language acceptance tests are complete
• If UX design is needed then the UX label is added
• If UX design is needed then the UX LoFi Design is completed, approved by the Product Manager, and added to the issue.
• A high-level solution (development approach for the code change) is added to the issue
• If prototype code is generated, it is linked to the issue.
• If tech briefs are generated, they are linked to the issue.

1B-DECOMP

• The work is broken into reasonable and consistent work items, and work item issues created. The work item issues are moved below the Done marker in the Decomp column.
• Estimate is updated for the work item issues.
• If there is a parent story epic issue, it is linked with the child work item issues and moved to the Epic column.
• If an issue is included in a product phase, then Release is selected.

1C-DESIGN

• The UX HiFi Design is complete, approved by the Product Manager, and added to the issue.
• Distilled acceptance tests are added to the issue.
• Automated testing criteria are added to issue.
• The solution is updated.

2A-CODE

• A pull request is opened, the automated pull request checks pass, and the change has been approved by other developers.
• The acceptance tests are finalized in "Given / When / Then" format.
• Regression test categories are identified on the issue. Categories should be referenced if any changes might affect them, even if the intended functionality is unchanged.

4A-UX REVIEW

• The pull request is merged into master, and the change branch is deleted.

5A-FUNCT VER

• All the acceptance tests for this issue pass.

5C-REGRESSION

• The test plan in testrail passes.
• Blessing from Product Manager that the release is good to go.

5D-READY TO BE DEPLOYED

• Production is updated and verified.
• All issues in the release are closed.

It would be useful if the error ID number and server-returned timestamp (X-App-Date?) were always visible on the 500 server error dialog. That would make it easier to look up the relevant logging.

What it looks like now:

SCENARIO Briefly describe the issue.

Steps to Reproduce the issue: Who, Where and How?

• Product:

• Accounts

• BigLearn

• Exercises

• CNX

• [ v] Tutor

• Payments

• Highlight

• Web

• User Type:

• Admin

• Content/QA

• Student

• [ v] Teacher

• N/A

• Username: Sarah LeGresley

• URL: https://tutor.openstax.org/course/4804/reading/43123

• Course number: PHSX 114:College Physics I

• Course type: PHSX 114:College Physics I

• Steps

Environment:

• Instance:
• [ v] production
• qa
• staging
• dev
• content
• demo
• OS:
• [ v] Mac OSX
• Windows
• Android
• iOS
• Browser:
• Chrome
• [ v] Firefox
• [ v] Safari
• Opera
• IE
• Edge
• Browser Version:

Possible impact on a User:
This non-sense date actually works as fine as 02/32==02/01 (mod 31)

Images or Screenshots (attach below)

SCENARIO Briefly describe the issue.

Say the student wants to multitask so they squeeze the browser's display to half of its size as before-
then:

1. the title "Time, Velocity and speed" got scrambled
2. the navigation arrows on both would block some content

Steps to Reproduce the issue: Who, Where and How?

• Product:

• Accounts

• BigLearn

• Exercises

• CNX

• [v] Tutor

• Payments

• Highlight

• [v] Web

• User Type:

• Admin

• Content/QA

• [v] Student

• Teacher

• N/A

• Username:

• URL:

• Course number:

• Course type:

• Steps

Environment:

• Instance:
• production
• qa
• staging
• dev
• [v] content
• demo
• OS:
• [v] Mac OSX
• Windows
• Android
• iOS
• Browser:
• Chrome
• [v] Firefox
• Safari
• Opera
• IE
• Edge
• Browser Version:
  65.0.1

Possible impact on a User:

Images or Screenshots (attach below)

Here's a suggestion for predictable names for commands that can be done on all the screens in tutor. I went through all the screens and tried to list off all the things a person can click. What do folks think?

/cc @openstax/tutor-fe @openstax/test

TODO

• add these to a documentation file

Teacher

These are largely teacher-specific screens

Calendar

assignmentType: 'HOMEWORK' | 'READING' | 'EXTERNAL' | 'EVENT'
relativeDate: 'TODAY' | 'TOMORROW' | 'FIRST'

• goToBook()
• goToForecast()
• goToScores()
• goToAddByType(assignmentType)
• goToAddByTypeFromNow(assignmentType, relativeDate)
• goToPreviousMonth()
• goToNextMonth()
• goToOpenByTitle(title)
• goToEditByTitle(title)
• goToReviewByTitle(title)

Popup

• selectPeriodByIndex(num)
• selectPeriodByTitle(title)
• goToReview()
• goToEdit()

CC-Dashboard

• goToScores()
• goToBook()
• selectPeriodByIndex(num)
• selectPeriodByTitle(title)

Builder

• edit({settings})

Scores

• selectDisplayAsNumber()
• selectDisplayAsPercent()
• selectBasedOnPossible()
• selectBasedOnAttempted()
• selectPeriodByIndex(num)
• selectPeriodByTitle(title)
• sortName()
• goToAssignmentByIndexes(row, column)
• downloadExport()

CC Commands:

• sortAssignmentScoreByIndex(index)
• sortAssignmentCompletedByIndex(index)
• sortAssignmentScoreByTitle(title)

HS Commands:

• goToForecastByName(studentName)
• goToReviewByIndex(index)
• goToReviewByTitle(title)

Review

stepStatus: 'ANY' | 'UNSTARTED' | 'COMPLETED' | 'CORRECT' | 'INCORRECT'

• goToBack()
• selectPeriodByIndex(num)
• selectPeriodByTitle(title)
• selectStepByIndex(index)
• selectStepByStatus(stepStatus)
• getExerciseByIndex(index)

Roster

• renameCourse(name)
• removeInstructorByIndex(index)
• removeInstructorByName(firstName, lastName=null)
• selectPeriodByIndex(num)
• selectPeriodByTitle(title)
• addPeriod(title)
• renamePeriod(title)
• removePeriod()
• changePeriodByName(firstName, lastName=null)
• removeStudentByIndex(index)
• removeStudentByName(firstName, lastName=null)

Student

These are largely student-specific screens

Dashboard

• showPastWork()
• showCurrentWork()
• goToPracticeByIndex(index)
• goToPracticeBySection('4.3')
• goToPracticeWeakest()
• goToForecast()
• goToBook()
• goToAssignmentByIndexe(index)
• goToAssignmentByType(assignmentType)
• goToAssignmentByTitle(title)

Both

These screens do the same thing for teachers and students

CourseSelect

courseType: 'ANY' | 'BIOLOGY' | 'PHYSICS' | 'CONCEPT_COACH'

• goToByType(courseType)
• goToByTitle(title)

User (the navbar on the top plus login/logout)

• goToHome()
• goToCourseHome()
• goToDashboard()
• goToForecast()
• goToBook()
• goToAccount()
• goToHelp()
• goToLogout()
• openHamburgerMenu()
• toggleHamburgerMenu()

Teacher-only:

• goToScores()
• goToRoster()

Forecast

Teacher-only:

• goToBack()
• selectPeriodByIndex(num)
• selectPeriodByTitle(title)

Student-only:

• goToBack()
• goToPracticeByIndex(index)
• goToPracticeBySection('4.3')

Task

• continue()
• selectStepByIndex(index)
• selectStepByStatus(stepStatus)
• selectCompletionStep()

Exercise

Teacher-only:

• toggleFreeResponse()

Student-only:

• enterFreeResponse(msg=null)
• selectAnswerByIndex(index)

SCENARIO Briefly describe the issue.

Images aligned in different ways- some in the center, others to the left
See screenshot:
(a)
this is the only case when one image is immediately followed by another image that is aligned differently

(b)
A group of pictures aligned in the same way, followed by some text paragraphs, and then another set of pics aligned in the other way

(c)
Suggestion:
The cnx version got all the images aligned in the center, which looks great

Steps to Reproduce the issue: Who, Where and How?

• Product:

• Accounts

• BigLearn

• Exercises

• CNX

• [v] Tutor

• Payments

• Highlight

• Web

• User Type:

• Admin

• Content/QA

• [v] Student

• Teacher

• N/A

• Username:

• URL:

• Course number:

• Course type:

• Steps

Environment:

• Instance:
• production
• qa
• staging
• dev
• [v] content
• demo
• OS:
• [v] Mac OSX
• Windows
• Android
• iOS
• Browser:
• Chrome
• [v] Firefox
• Safari
• Opera
• IE
• Edge
• Browser Version:
  65.0.1

Possible impact on a User:

Images or Screenshots (attach below)

Should this be? Going back to the calendar would seem more intuitive to me.

It should be https://openstaxcc.zendesk.com/hc/en-us

Link to media currently is

• jumpy
  • addressed by #739
• removes user from reading context
  • addressed by #767

This way there are no courses with 0 instructors

When Firefox 46 is released, the JavaScript based setMaxImageWidth fix from #974 should be removed.

Details

Image max-width was not working in Firefox.

Update

02/10/2016

While the Firefox bug has been marked as RESOLVED, you can still see it's currently an issue in Firefox 44.01 in this JS Fiddle. Based on this bug report linked in this comment for the 975632, the fix will be released in Firefox 46, slated for release April 19th. Based on a quick test on Nightly which is at 46.0a2, the bug is indeed fixed.

44.01

46.0a2

Description

Allow user to tab through paragraphs in the reading with Chrome plugin. Verify that this is a bug, and that we need to use Chrome box. Report back with correct keys.

#Acceptance
User can tab through paragraphs

List of all User Errors Automated as well as reported to CS!

P.S. Add user errors in comments along with the case#

SCENARIO _View in book icon in Highlight summary page works incorrectly for images

Steps to Reproduce the issue: _
Highlight an image
From the Highlights SUmmary page, click on the 'View in book' icon
User is taken to the top of the page in which image is present and not to the image location

• Product:
  [ ] Accounts [ ] BigLearn
  [ ] Exercises [ ] CNX
  [ ] Payments [ ] Tutor
  [ ] Highlight [ ] Web

• User Type:
  [ ] Admin [ ] Content/QA
  [X] Student [X] Teacher
  [ ] N/A

• Username:

• URL:

• Course number:

• Course type:

• Steps

Environment:

• Instance:
  [ ] production [X] qa [ ] staging
  [ ] dev [ ] content [ ] demo
• OS:
  [ ] Mac OSX [ ] Windows
  [ ] Android [ ] iOS
• Browser:
  [ ] Chrome [ ] Firefox [X] Safari
  [ ] Opera [ ] IE [ ] Edge
• Browser Version:

Possible impact on a User:

Images or Screenshots (attach below)

SCENARIO Briefly describe the issue.

There are several cases when the lack of space among the contents may confuse the readers-

(a) between the text box and the next subtitle

For instance, we'd better insert a space between the text box and the next subtitle “Percent Uncertainty”

Also, check out screenshots attached below:

(b) at the beginning of every problem set, usually the title

See the pictures attached below:

This is purely an aesthetic issue and less critical as compared with (a).

However, for both (a) and (b), I do recommend inserting a space at the appropriate place.

(c) the subtitle between two text boxes

This is a clarity issue since the subtitle usually belongs to the second text box.

What I recommend here is to apply the format in the CNX version:

such that it's clear that the second chunk of info is grouped together.

Steps to Reproduce the issue: Who, Where and How?

• Product:

• Accounts

• BigLearn

• Exercises

• [v] CNX

• [v] Tutor

• Payments

• Highlight

• [v] Web

• User Type:

• Admin

• Content/QA

• [v] Student

• Teacher

• N/A

• Username: teacher02

• URL: https://tutor-qa.openstax.org/book/350/section/1.2

• Course number:

• Course type: Physics AP

• Steps

Environment:

• Instance:
• production
• qa
• staging
• dev
• [v] content
• demo
• OS:
• [v] Mac OSX
• Windows
• Android
• iOS
• Browser:
• Chrome
• [v] Firefox
• Safari
• Opera
• IE
• Edge
• Browser Version:
  65.0.1

Possible impact on a User:

Images or Screenshots (attach below)

Pop up a modal (or similar) for when a 500 error occurs, with standard "An error occurred" messaging, the returned error_id, and a help desk link? We're going to be 500'ing if the practice widget ends up being empty (which has happened and shouldn't be happening), and it'd be nice if the 500 showed up in more than just the console.

Upgrade Dependencies

• react
  • [email protected]
    • we can do dynamic chunks really easily: https://github.com/rackt/react-router/blob/master/docs/guides/advanced/DynamicRouting.md
    • uses history package instead of Navigation
  • react-bootstrap
    • OverlayTrigger: react-bootstrap/react-bootstrap#855
    • trigger='manual' removed
    • NavBrand formatting
    • DropdownMenu, MenuItem update jsx format / props
  • fixed-data-table
  • react-calendar
• standardize selenium helper names
• remove jQuery
• update phantomjs to 2.x
• replace underscore with lodash
• upgrade time stuffs -- #1055
  • moment
  • twix
  • react-datepicker

Refactor Code

• use classnames
• remove bootstrap-material-design
• remove lint warnings from cjsx files
• moment
• increase code coverage
• replace flux
• ES6?!?!?!?!??!?!?!

SCENARIO Briefly describe the issue.

Assignment Due Date rendered as 02/32/2019 in Tutor

Steps to Reproduce the issue: Who, Where and How?

Signed in as instructor Sarah LeGresley and tried to edit the reading assignment on Feb 1st in her course Spring 2019 PHSX 114:College Physics I

• Product:

• Accounts

• BigLearn

• Exercises

• CNX

• [ v] Tutor

• Payments

• Highlight

• Web

• User Type:

• Admin

• Content/QA

• Student

• [v ] Teacher

• N/A

• Username: Sarah LeGresley

• URL: https://tutor.openstax.org/course/4804/reading/43123

• Course number: PHSX 114:College Physics I

• Course type:

• Steps

Environment:

• Instance:
• [ v] production
• qa
• staging
• dev
• content
• demo
• OS:
• [ c] Mac OSX
• Windows
• Android
• iOS
• Browser:
• Chrome
• [ v] Firefox
• [ v] Safari
• Opera
• IE
• Edge
• Browser Version:

Possible impact on a User:

worked just as fine as 02/32/2019==02/01/2019 (mod 31)...
this problem appeared in multiple assignments

Images or Screenshots (attach below)

An instructor came in via chat bc Tutor doesn't have a term that fits the her course that begins in November and ends in February. The Winter term is 1/1 to 4/30 and the Fall term is 7/1 to 12/31. Is there a way to change the default Winter term to include Christmas break (at least).

Case #28342

Does not happen with the other buttons...

Writing this as an issue vs a napkin note so so people can comment.

We can write selenium helpers as objects, or as a collection of structured functions.

An simplified version of an object oriented helper would be something like:

class ProgressGuide extends PanelElement
  constructor: (test) ->
    @element = test.driver.findElement(css: '.student-dashbaord')

  getTitle: ->
    @element.findElement(css: '.title').then (el) ->
      el.getText()

class StudentDashboard
  constructor: (test) ->
    @element = test.driver.findElement(css: '.student-dashbaord')

  getProgressGuide: ->
    new Promise (resolve, reject) =>
      @element.findElement(css: '.progress-guide').then (el) =>
        resolve(new ProgressGuide(@test, el))

module.exports = StudentDashboard

To use this in a spec you could:

StudentDashboard = require 'helpers/student-dashboard'

@it 'has the proper title', ->
    dash = new StudentDashboard(@)
    dash.getProgressGuide().then (guide) ->
      guide.getTitle().then (title) ->
        expect(title).to.equal('Progress Guide')

As you can see, the object oriented helper yields different things from it's promises. StudentDashboard.getProgressGuide() yields an instance of a ProgressGuide object, while the promise returned by getTitle yields a string.

Phil suggested we could employ a different method that might be less confusing. In his suggestion, each getter method would return a selenium element object. That way there would be no confusion over what a helper would yield.

Code written in that manner would look more like:

getProgressGuide = (test) ->
  test.driver.findElement(css: '.student-dashbaord')

ProgressGuide =
  getTitle: (el) ->
    @element.findElement(css: '.title')



module.exports = {getProgressGuide, ProgressGuide}

and could be used as:

StudentDashboard = require 'helpers/student-dashboard'

@it 'has the proper title', ->
  StudentDashboard.getProgressGuide(@).then (guideEl) ->
    StudentDashboard.ProgressGuide.getTitle(guideEl).then (title) ->
      title.getText().then (title) -> # <- title is an Element, getText is selenium
        expect(title).to.equal('Progress Guide')

Thoughts on the two approaches? @philschatz if I've misrepresented your approach please correct me

refs #931

Right next to "Report an error"

Bug was introduced in #980 ( https://github.com/openstax/tutor-js/pull/980/files#r54943384 )

"That added a .tab-item-period to the specifier, but the mixin is used elsewhere so other uses lost the background color"

The issue this fixes is hard to reproduce. Steps are:

1. Answer a free response
2. Choose a multiple choice answer
   • Must be while free response is still saving/while we have not received a response back from the server
3. A response similar to:
   {"status":422,"errors":[{"code":"free_response_is_required","message":"Free response is required"}]}
   is received

Prev, Next re-opens TOC

This is annoying, prev next re-opens TOC even though it was manually closed

TOC does not auto-close

Also, for a small window, TOC should close automatically when a TOC item is navigated to

PheT sims in physics sections 22.2 and 22.11 homework questions not displaying in question library from teacher view

SCENARIO Briefly describe the issue.
Navigation Icon Blocks Highlighting Activities

Steps to Reproduce the issue:
I was trying to highlight things when doing the homework as a student.
Because of the navigation icon on the left side, I can only either highlight one single sentence as I did in the first screen or the entire thing as in the second one.

• Username: student04

• URL: https://tutor-qa.openstax.org/enroll/916584/Alicia-College-Physics-Spring-2019

• Course number:

• Course type: College Physics

• Steps

Environment:

• Instance:
• production
• qa
• staging
• dev
• [v] content
• demo
• OS:
• [v] Mac OSX
• Windows
• Android
• iOS
• Browser:
• Chrome
• [v] Firefox
• Safari
• Opera
• IE
• Edge
• Browser Version:
  65.0.1

Possible impact on a User:

Images or Screenshots (attach below)

SCENARIO Briefly describe the issue.
Due date on assignments for courses created before reimport are showing incorrect numbers in date field.
the screenshot is from the old course and the due date shows as 12/344/2018
Happens in both QA and prod

Steps to Reproduce the issue: Who, Where and How?

• Product:

• Accounts

• BigLearn

• Exercises

• CNX

• Tutor

• Payments

• Highlight

• Web

• User Type:

• Admin

• Content/QA

• Student

• Teacher

• N/A

• Username: teacher01 (tutor-qa)

• URL: https://tutor-qa.openstax.org/course/2052/reading/37595

• Course number:

• Course type:

• Steps

Environment:

• Instance:
• production
• qa
• staging
• dev
• content
• demo
• OS:
• Mac OSX
• Windows
• Android
• iOS
• Browser:
• Chrome
• Firefox
• Safari
• Opera
• IE
• Edge
• Browser Version:

Possible impact on a User:

Images or Screenshots (attach below)

Since many of us work remote, an async discussion might be easier.

Thoughts on the easiest way to improve confidence when deploying?

Some options (add more!):

• add a coveralls.io webhook
  • easy to drop in
• selenium scripts
  • utility methods would help QA folks
  • would better test how code works between components
  • downside: might require starting up tutor (too slow for travis?) or more mock JSON files (not too bad)
• Selenium + coveralls: see #924 for how-to

Here is the unit test code coverage as of @a68f082. By changing the reporter in test/config/karma-coverage.config.coffee#L12 (adding coverageReporter: { type: 'lcov', dir: './coverage'} I got the following report: https://openstax.github.io/tutor-js/coverage/

@darrenmason @jivewise @nathanstitt @pandafulmanda ?

When selecting today for the open date I get the following error:

Thoughts @jivewise or @pandafulmanda ?

It may be worthwhile to all fonts locally (or all remotely). This stems from #85.

Some links from @Fredasaurus:

• http://stackoverflow.com/questions/8966740/how-to-host-google-web-fonts-on-my-own-server
• http://michaelboeke.com/blog/2013/09/10/Self-hosting-Google-web-fonts/

We decided to use the MathJax CDN (@lakshmivyas) and maybe some discussion should be had on CSS fonts.

Course roster sort is case-sensitive but likely should not be. Right now, a last name of "del Martos" would come after all other names starting with a capital letter.

I created a new user (without a course) and logged in and was taken to the Physics I dashboard, even tho he has no roles.

http://localhost:3001/api/courses.json

returns

[
  {
    id: "1",
    name: "Physics I",
    roles: [ ]
  }
]

Where can we send these kind of anonymous users that will sign up but not be in a course?

SCENARIO Briefly describe the issue.
Highlight Icon doesn't show up when supposed to.
See the inconsistency between the two pictures attached below.

Steps to Reproduce the issue: Who, Where and How?
Highlight Icon doesn't show up when supposed to when doing the homework as a student.
See the inconsistency between the two pictures attached below.

• Product:

• Accounts

• BigLearn

• Exercises

• CNX

• [v] Tutor

• Payments

• Highlight

• Web

• User Type:

• Admin

• Content/QA

• [v] Student

• Teacher

• N/A

• Username: teacher04

• URL: https://tutor-qa.openstax.org/enroll/916584/Alicia-College-Physics-Spring-2019

• Course number:

• Course type: College Physics

• Steps

Environment:

• Instance:
• production
• qa
• staging
• dev
• [v ] content
• demo
• OS:
• [v] Mac OSX
• Windows
• Android
• iOS
• Browser:
• Chrome
• [ v] Firefox
• Safari
• Opera
• IE
• Edge
• Browser Version:
  65.0.1

Possible impact on a User:

Images or Screenshots (attach below)

Since is_active: false students are not displayed in the roster, there is no way for a teacher to undrop students.