Unable to run fxa-content-server tests about fxa-content-server HOT 18 CLOSED

This did work for me:
I "git cloned" and "npm installed" fxa-auth-server and fxa-content-server

In terminal 1: I am running the fxa-auth-server
In terminal 2: I am running java for the Selenium jar
In terminal 3: I am running the fxa-content-server
In terminal 4: from the fxa-content-server, I ran the following:
Integration tests between the FAB and fxa-auth-server: npm test
Functional tests: npm run-script test-functional

The console output for fxa-auth-server looks good.
The console output for Selenium looks good.
The console output for fxa-content-server is very minimal, but looks good
Test output was good:

$ npm test
> [email protected] test /Users/James/fxa-content-server
> node node_modules/intern/runner.js config=tests/intern suites=tests/tdd 
Defaulting to "runner" reporter
Listening on 0.0.0.0:9090
Initialised firefox 25.0.1 on MAC
No coverage report available
firefox 25.0.1 on MAC: 0/10 tests failed
=============================== Coverage summary ===============================
Statements   : 100% ( 0/0 )
Branches     : 100% ( 0/0 )
Functions    : 100% ( 0/0 )
Lines        : 100% ( 0/0 )
================================================================================
TOTAL: tested 1 platforms, 0/10 tests failed


$ npm run-script test-functional
> [email protected] test-functional /Users/James/fxa-content-server
> node node_modules/intern/runner.js config=tests/intern_functional
Defaulting to "runner" reporter
Listening on 0.0.0.0:9090
Initialised firefox 25.0.1 on MAC
No coverage report available
firefox 25.0.1 on MAC: 0/2 tests failed
=============================== Coverage summary ===============================
Statements   : 100% ( 0/0 )
Branches     : 100% ( 0/0 )
Functions    : 100% ( 0/0 )
Lines        : 100% ( 0/0 )
================================================================================
TOTAL: tested 1 platforms, 0/2 tests failed

from fxa-content-server.

Hey @pdehaan, I'll take a look, we might just need to adjust the npm test command.

from fxa-content-server.

@jbonacci Odd, those are the same steps I did locally (I even recloned the repos this morning from the Mozilla repo and not my fork).

Nothing interesting in the fxa-auth-server tab, the selenium tab shows the following output:

OpenGL version detected: 210
OpenGL version detected: 210

at org.openqa.selenium.firefox.internal.NewProfileExtensionConnection.start(NewProfileExtensionConnection.java:106)
... 19 more
11:52:19.478 WARN - Exception: Unable to connect to host 127.0.0.1 on port 7055 after 45000 ms. Firefox console output:
*** LOG addons.manager: Application has been upgraded
*** LOG addons.xpi: startup
*** LOG addons.xpi: Skipping unavailable install location app-system-share
*** LOG addons.xpi: Ignoring file entry whose name is not a valid add-on ID: /var/folders/qc/g_f0szh95k1fzph34rhmf5jw0000gp/T/anonymous1407984438819059861webdriver-profile/extensions/webdriver-staging
*** LOG addons.xpi: checkForChanges
*** LOG addons.xpi-utils: Opening XPI database /var/folders/qc/g_f0szh95k1fzph34rhmf5jw0000gp/T/anonymous1407984438819059861webdriver-profile/extensions.json
*** LOG addons.xpi: New add-on [email protected] installed in app-profile
*** Blocklist::_loadBlocklistFromFile: blocklist is disabled
*** LOG addons.xpi-utils: Make addon app-profile:[email protected] visible
*** LOG DeferredSave/extensions.json: Save changes
*** LOG DeferredSave/extensions.json: Save changes
*** LOG addons.xpi: New add-on {972ce4c6-7e08-4474-a285-3208198ce6fd} installed in app-global
*** LOG addons.xpi-utils: Make addon app-global:{972ce4c6-7e08-4474-a285-3208198ce6fd} visible
*** LOG DeferredSave/extensions.json: Save changes
*** LOG DeferredSave/extensions.json: Save changes
*** LOG addons.xpi: Updating database with changes to installed add-ons
*** LOG addons.xpi-utils: Updating add-on states
*** LOG addons.xpi-utils: Writing add-ons list
*** LOG DeferredSave/extensions.json: Starting timer
*** LOG DeferredSave/extensions.json: Starting write
*** LOG DeferredSave/extensions.json: Write succeeded
*** LOG addons.xpi-utils: XPI Database saved, setting schema version preference to 15
*** LOG addons.manager: shutdown
*** LOG addons.xpi: shutdown
*** LOG addons.xpi-utils: shutdown
*** LOG addons.xpi: Notifying XPI shutdown observers
*** LOG addons.manager: Async provider shutdown done
*** LOG addons.xpi: startup
*** LOG addons.xpi: Skipping unavailable install location app-system-share
*** LOG addons.xpi: Ignoring file entry whose name is not a valid add-on ID: /var/folders/qc/g_f0szh95k1fzph34rhmf5jw0000gp/T/anonymous1407984438819059861webdriver-profile/extensions/webdriver-staging
*** LOG addons.xpi: checkForChanges
*** LOG addons.xpi: No changes found
OpenGL version detected: 210
OpenGL version detected: 210

What's interesting from your results is that it used FF 25.0.x but I only have FF 26, 27, and 28.

from fxa-content-server.

Actually, Edwin and I sat down and could repro [my problems] on his machine.
Digging deeper, here's what we found:

• It didn't work for me w/ FF26 on 10.8.
• It didn't work for Edwin w/ FF26 on 10.9.
• It DID work for Edwin w/ FF24 and 25 (after we disabled the profile chooser).
• It didn't work for Edwin w/ FF26 after we disabled the profile chooser.

So it looks like the stalling tests may be related to browser versions (newer than 25) and having the profile chooser dialog pop up.

I'll see if there is a newer Selenium .jar file which works w/ FF26+.

from fxa-content-server.

Wow, yea, I forget the default for Selenium is to grab your "default" version of FF. Mine just happens to be FF 25.

from fxa-content-server.

looks like webdriver peeps know about this and will have a fix in v36

http://code.google.com/p/selenium/issues/detail?id=6323

from fxa-content-server.

I have 2.35.0, the latest seems to be 2.37.0.
Now I am wondering what the rest of QA is using for the jar version and their default FF version...

from fxa-content-server.

OK, so then 2.36 and 2.37.
I am going to grab 2.37 and just run this all again...

from fxa-content-server.

One note here - Sauce Labs Selenium versions: https://saucelabs.com/docs/additional-config#selenium-version
2.37.0 is available for use

from fxa-content-server.

Confirmed, this works for me in 2.37.0 on OS X 10.8.x and FF26 (with profile chooser enabled).
I'll update the README file to mention 2.37.0 and dig up the direct download link.
UPDATE: http://selenium.googlecode.com/files/selenium-server-standalone-2.37.0.jar

$ npm test

> [email protected] test /Users/pdehaan/dev/fxa-content-server_moz/fxa-content-server
> node node_modules/intern/runner.js config=tests/intern suites=tests/tdd

Defaulting to "runner" reporter
Listening on 0.0.0.0:9090
Initialised firefox 26.0 on MAC
No coverage report available
firefox 26.0 on MAC: 0/10 tests failed

=============================== Coverage summary ===============================
Statements   : 100% ( 0/0 )
Branches     : 100% ( 0/0 )
Functions    : 100% ( 0/0 )
Lines        : 100% ( 0/0 )
================================================================================
TOTAL: tested 1 platforms, 0/10 tests failed

from fxa-content-server.

Yes, well, now mine fails
FF 25.0.1 using 2.37.0
/me shrugs

Defaulting to "runner" reporter
Listening on 0.0.0.0:9090
Initialised firefox 25.0.1 on MAC
Test main - login - create account form FAILED on firefox 25.0.1 on MAC:
Error: Validates false login: expected '' to equal 'Try another email or Create an account'
Validates false login: expected '' to equal 'Try another email or Create an account'
No coverage report available
firefox 25.0.1 on MAC: 1/2 tests failed

from fxa-content-server.

And this:
Defaulting to "runner" reporter
Listening on 0.0.0.0:9090
Initialised firefox 25.0.1 on MAC
Error: Error: CORS request rejected: http://127.0.0.1:9000/v1/account/create at http://localhost:9090/static/js/gherkin.js:3586
window.onerror@http://localhost:9090/__intern/client.js:62

No coverage report available
firefox 25.0.1 on MAC: 0/0 tests failed

from fxa-content-server.

Notes:
Terminal 1: 20:48:35.001Z INFO fxa-auth-server: running on http://127.0.0.1:9000
Terminal 2: Dec 3, 2013 12:49:06 PM org.openqa.grid.selenium.GridLauncher main
INFO: Launching a standalone server
Terminal 3: FAB: Firefox Account Bridge listening on port 3030
Terminal 4: the actual tests

This all worked before (see my previous comments using FF 25.0.1 and jar 2.35.0)

from fxa-content-server.

@jbonacci Deeper down the rabbit hole we go!

OK, so it works for me w/ today's fxa-auth-server and fxa-content-server and selenium 2.37.0 w/ FF26 on OSX 10.8.

I'll try installing FF25 (current stable release channel) and try to repo w/ the 2.35.0 and 2.37.0 .jar files.

Looks like my goals of updating the README from 2.35 to 2.37 isnt so easy after all.

from fxa-content-server.

Ah interesting, what versions that we are targeting to test anyway?
Should fxa be FF26+ or also include FF25?

from fxa-content-server.

Not sure - that sounds like a question for the product team.
For testing purposes, we should be supporting FF25 - FF28.

Does anyone remember how to force Selenium to use a different version?
I have all four on my Mac, and it will only run against FF 25.
If I remove that, Selenium barfs...

from fxa-content-server.

Reinstalled everything.
Now I get good results using FF 25 and 2.35/2.37 jars

I see one error from the fxa-content-server though:
FAB ERR: error: TypeError: Cannot read property 'encrypted' of null

from fxa-content-server.

Closing. Reopen or file a new bug if we see this again.

from fxa-content-server.