component-driven / cypress-axe Goto Github PK
View Code? Open in Web Editor NEWTest accessibility with axe-core in Cypress
License: MIT License
Test accessibility with axe-core in Cypress
License: MIT License
I am creating a new issue for #7 as the original owner closed it despite the issue not being resolved. If you are having issues getting cypress-axe to work with Typescript/Webpack, please look at that issue for solutions.
Hey!
In some cases, the full axe-core results object is needed (for logging / tracking or other uses)
It would be helpful to have another callback that returns the full results object (similar to how violationsCallback returns the violations object).
I submitted a PR for this - please consider it :)
#68
I started using cypress-axe
to catch a11y issues when running end to end tests automatically, but I am finding it challenging to get the tests to pass. I get two errors which I believe are false positives:
I use react-axe
, which prints a11y issues to browser console in development mode, but I don't see these two issues reported by that package. I tried auditing using Axe Chrome extension, but I do not see these.
I would be happy to provide more information, but at this point, I believe it to be a bug with cypress-axe.
First of all: Nice project, great job ;-)
Is there a reason why there is no license?
I see a couple of issues opened to upgrade axe core version, which is bundled in this library. (#36, #58)
To avoid this type of request in the future, I think we can make this awesome library more flexible by allowing users to use any axe core version.
I'm proposing to add a parameter to the injectAxe method, so users can specify their axe core source/version to overwrite the default version.
Sample scenario: Assuming the user wants to use the latest axe core
npm install -D axe-core@latest
cy.readFile('node_modules/axe-core/axe.min.js', 'utf-8').then((content) => {
cy.injectAxe(content);
cy.checkA11y();
})
With Cypress 4.0 released, could this be added now to peerDeps? I don't know if other changes would be needed. Thanks for allowing us to test accessibility with cypress!
I'm seeing the following error when running cy.injectAxe
in my Cypress tests:
Timed out retrying after 4000ms: cy.readFile("0764") failed because the file does not exist at the following path:
/home/megan/voicethread/code/frontend/wizard/0764
Because this error occurred during a before each hook we are skipping the remaining tests in the current suite
node_modules/cypress-axe/dist/index.js:16:8
14 | exports.configureAxe = exports.injectAxe = void 0;
15 | exports.injectAxe = function () {
> 16 | cy.readFile(require.resolve('axe-core/axe.min.js')).then(function (source) {
| ^
17 | return cy.window({ log: false }).then(function (window) {
18 | window.eval(source);
19 | });
require.resolve('axe-core/axe.min.js')
seems to evaluate to "0764"
instead of the intended JS.
This issue seems to be connected to Cypress's Webpack Preprocessor, which I'm using to handle importing files with absolute path aliases.
const webpack = require('@cypress/webpack-preprocessor')
module.exports = (on, config) => {
on('file:preprocessor', webpack({
webpackOptions: require('@vue/cli-service/webpack.config'),
watchOptions: {},
// Add the ability to use aliases like @shared
resolve: {
alias: require('../../../../aliases.config').webpack
}
}))
return Object.assign({}, config, {
fixturesFolder: 'tests/e2e/fixtures',
integrationFolder: 'tests/e2e/specs',
screenshotsFolder: 'tests/e2e/screenshots',
videosFolder: 'tests/e2e/videos',
supportFile: 'tests/e2e/support/index.js'
})
}
When I remove the on('file:preprocessor')
listener from my plugins.js
, file, the cy.injectAxe
command succeeds.
Node version: 12.13.1
Cypress version: 6.3.0
@cypress/webpack-preprocessor version: 5.5.0
Is there another way to see the a11y failures than through the console? I'm finding it somewhat time consuming to identify the violating DOM node, it'd be neat if you could use regular Cypress functionality like highlighting the DOM node through the UI, etc.
I have this in my support/index.ts
my tsconfig.json looks like this:
{
"compilerOptions": {
"lib": [
"dom",
"es2015",
"es5",
"es6",
"es2015"
],
"target": "es5",
"module": "commonjs",
"noImplicitAny": false,
"strictNullChecks": true,
"noImplicitThis": true,
"alwaysStrict": true,
},
"files": [
"./support/index.d.ts"
]
}
I have this test:
describe("TypeScript", () => {
it("checks shape of an object", () => {
cy.visit('http://localhost:6001')
cy.injectAxe();
});
})
It fails with
ReferenceError: exports is not defined
at axeFunction (eval at cy.window.then.window (http://localhost:6001/__cypress/tests?p=cypress/support/index.ts-538:111:12), :16:14)
at eval (eval at cy.window.then.window (http://localhost:6001/__cypress/tests?p=cypress/support/index.ts-538:111:12), :15287:3)
at eval ()
Why would this be?
axe-core has rules provided at https://github.com/dequelabs/axe-core/tree/develop/lib/rules
but could not find an example custom json to append custom rules. What I want to do is to manually test a site using Trusted Tester Test Process, and add the test to the custom list of tests with expected / actual type assertions, so as the custom tests grow in size the amount of time needed to test against trusted tester tests, manually is reduced.
Hello,
Love the product! This is a great idea and way to manage a11y development, creating a much tighter feedback loop.
I wanted to request support for configuring axe rule enforcement - some rules are inappropriately enforced in certain scenarios, leading to false positives. Ideally in cypress/support/index.js
a config file could be passed in, leveraging axe.configure()
.
This may already be possible! If so, just some simple docs with an example would suffice.
Whenever I try to use cy.injectAxe()
in a test, Cypress gets stuck at the step immediately prior. The Cypress test runner itself is locked and must be force-quit.
Here is code to reproduce the problem.
context('Problem with cy.injectAxe()', function () {
beforeEach (function () {
// These are not real credentials.
// Please sign up (for free) at www.vidyard.com, verify your email address, then update these accordingly before running.
const email = '[email protected]'
const password = 'password123!'
cy.visit('https://secure.vidyard.com/user/sign_in')
cy.get('#username').type(email)
cy.get('#password').type(password)
cy.get('#sign-in').click()
cy.wait(10000) // Just to make sure the page is fully loaded.
})
it('Accessibility Test', function () {
// When the lines below are commented out, there is no issue.
// When the lines are let to run, the Cypress Test Runner gets stuck on the cy.wait step.
// Cypress will be completely frozen and must be force quit.
cy.injectAxe()
cy.checkA11y()
})
})
I am using Cypress version 4.12.1 and cypress-axe
version 0.8.1 on macOS 10.14.6 with Chrome 84.0.4147.105 and Firefox 79.0.
We are trying to use a custom matches for some of the axe rules to remove a few false positives that are currently being thrown. The documentation for axe-core states that a string with the relative path to the JS file containing the custom matches function must be used (https://github.com/dequelabs/axe-core/blob/develop/doc/developer-guide.md#matches-function).
The problem we are having is what is the path relative too as nothing we have tried has worked.
Also is there any way of importing the original matches function used by axe core ie (https://github.com/dequelabs/axe-core/blob/develop/lib/rules/color-contrast-matches.js)
I have a modal that overlays the screen:
(note backdrop darkens)
I'm testing for a11y like so: cy.checkA11y(".modal", null, terminalLog);
But always get this contrast error:
failureSummary: "Fix any of the following:
Element has insufficient color contrast of 2.92 (foreground color: #979797, background color: #ffffff, font size: 13.5pt (18px), font weight: normal). Expected contrast ratio of 4.5:1"
The font size is 18px, the text is #000 and the background is #FFF. I'm not sure how it's getting this #979797 value. This bug only happens when I am checking my modal element for a11y, so I think something is going wrong when I pass in the element I want to check.
Getting this error when using 0.9.1 with axe-core 4.0.2 as a peer dependencies
ReferenceError: exports is not defined
at axeFunction (eval at <anonymous> (http://172.25.25.197:3000/__cypress/tests?p=cypress/support/index.js), <anonymous>:24:14)
at eval (eval at <anonymous> (http://172.25.25.197:3000/__cypress/tests?p=cypress/support/index.js), <anonymous>:27630:3)
at eval (<anonymous>)
With 0.8.1 even after you add the import command to support/index.js the axe commands are not detected.
cy.injectAxe()
cy.injectAxe()
cy.checkA11y()
Those commands are not detected by cypress..
As mentioned here, the latest version of axe-core fixes a CSP issue where injectAxe() doesn't work with a strict content security policy that doesn't allow unsafe-eval for scripts.
I'll submit a PR for this.
suddenly getting this error with no changes to my code. Using version 0.10.0:
have tried deleting dist and node_modules folder, running npm run update
and built new dist
Module not found: Error: Can't resolve 'axe-core/axe.min.js' in /path/node_modules/cypress-axe/dist
Hi,
I am trying to follow the instructions as documented.
When doing cypress open, I have the following error:
ReferenceError: t is not defined
This is happening when trying to eval(axe.source).
It appears that axe is not defined inside the eval.
I am not sure why.
I am using these versions:
"axe-core": "4.0.1",
"cypress": "4.11.0",
"cypress-axe": "0.8.1",
Using this in my cypress test:
cy.injectAxe();
cy.checkA11y({
exclude: [
[['#onetrust-consent-sdk']]
]
});
But getting these error:
SyntaxError
Failed to execute 'querySelectorAll' on 'Document': '[object Object]' is not a valid selector.
It seems like the first param for checkA11y
should not be an object, yet looking at the documentation and examples here it should work:
Any ideas? My goal is to ignore a11y checks for the div specified by an ID.
My test file:
context('app', () => {
beforeEach(() => {
cy.server();
cy.route('GET', '/api/exercises/core/exercises/*', 'fixture:exercises');
cy.visit('/exercises/3bc94905-4b8a-4d67-815e-0652afaf22c2');
});
it('has no a11y violations', () => {
cy.injectAxe();
cy.checkA11y();
});
});
the output (note the input field shown in element explorer)
A11Y ERROR! landmark-one-main on 1 Node
A11Y ERROR! page-has-heading-one on 1 Node
shows nothing about missing label on input, am i doing something wrong? ive tried adding arguments to the checkA11y
call to make it more specific but then i get no errors at all
Hi,
I noticed that checkA11y failed when called after using the cy.clock(...)
function to set a date
I get the following error when trying to call checkA11y :
cy.then() timed out after waiting 4000ms.
Your callback function returned a promise that never resolved.
The callback function was:
function(a){i(e)&&(e=void 0),i(t)&&(t=void 0),i(n)&&(n=void 0);var o=t||{},s=o.includedImpacts,u=r(o,["includedImpacts"]);return a.axe.run(e||a.document,u).then((function(e){var t=e.violations;return s&&Array.isArray(s)&&Boolean(s.length)?t.filter((function(e){return s.includes(e.impact)})):t}))}
Running cypress-axe 0.8.1
and cypress 4.12.1
Hey there. I am experience what I think is a strange issue. I am testing out the accessibility on my website built with Gatsby (React.js). I have Cypress set up as per their docs with this library and axe-core. When I run the the following I get no errors on a page that has a footer which I know is rendering links that will fail the color contrast test. I know this because I have react-axe installed as well and it is reporting the color contrast problem in the console.
However because react-axe does not lead to a nice automation testing work flow I would like to use cypress to find these color contrast issues.
What could be causing cypress-axe / axe-core not picking it up with testing but react-axe picks it up as expected?
Sample test
describe('Accessibility tests', () => {
beforeEach(() => {
cy.visit('http://localhost:8000')
cy.injectAxe()
})
it('Has no detectable accessibility violations on load', () => {
cy.wait(1000)
cy.checkA11y()
})
})
In https://github.com/avanslaars/cypress-axe/blob/master/src/index.js#L3, I see that the file is directly loaded from node_modules
using fs
I see that axe-core
exposes a source
parameter. Can that be used instead?
i.e.
import axe from 'axe-core';
export const injectAxe = () => {
cy.window({ log: false }).then(window => {
window.eval(axe.source)
})
}
Is it a concern if it is unminified?
Firstly, cypress-axe is brilliant! Previously I was using axe-core with webdriverio, and it was painfully slow! However, one frustration with cypress-axe is the use of node assert. Elsewhere in our tests I'm using the chai library bundled into Cypress, which I have also extended to provide soft assertions (ie, continue on assertion fail) based on code found here:https://stackoverflow.com/questions/55868107/does-cypress-support-soft-assertion
I have modified cypress-axe locally to use chai.softExpect, and it behaves as I'd like. this allows me to run multiple a11y checks against different pages / page sections in the same test.
Might it be possible to have some sort of configuration option that allowed the user to specify which assertion library / method to use? Or something similar?
I'm using cypress-axe to do some accessibility testing, but the errors I'm getting are not clear enough for me to act on... Is there a way to get more detailed information here?
6 passing (23s)
22 pending
2 failing
1) Accessibility Tests Global Component Assessments Desktop Assessments Petition Petition form should be accessible.:
AssertionError: 1 accessibility violation was detected: expected 1 to equal 0
at Function.fn.(anonymous function) [as equal] (http://localhost:8000/__cypress/runner/cypress_runner.js:86181:34)
at getRet (http://localhost:8000/__cypress/runner/cypress_runner.js:89187:16)
at tryCatcher (http://localhost:8000/__cypress/runner/cypress_runner.js:139407:23)
at Function.Promise.attempt.Promise.try (http://localhost:8000/__cypress/runner/cypress_runner.js:136682:29)
at Context.thenFn (http://localhost:8000/__cypress/runner/cypress_runner.js:89201:23)
at Context.then (http://localhost:8000/__cypress/runner/cypress_runner.js:89525:21)
at Context.<anonymous> (http://localhost:8000/__cypress/runner/cypress_runner.js:100860:21)
at http://localhost:8000/__cypress/runner/cypress_runner.js:100381:33
at tryCatcher (http://localhost:8000/__cypress/runner/cypress_runner.js:139407:23)
at Promise._settlePromiseFromHandler (http://localhost:8000/__cypress/runner/cypress_runner.js:137343:31)
at Promise._settlePromise (http://localhost:8000/__cypress/runner/cypress_runner.js:137400:18)
at Promise._settlePromiseCtx (http://localhost:8000/__cypress/runner/cypress_runner.js:137437:10)
at Async../node_modules/bluebird/js/release/async.js.Async._drainQueue (http://localhost:8000/__cypress/runner/cypress_runner.js:134137:12)
at Async../node_modules/bluebird/js/release/async.js.Async._drainQueues (http://localhost:8000/__cypress/runner/cypress_runner.js:134142:10)
at Async.drainQueues (http://localhost:8000/__cypress/runner/cypress_runner.js:134016:14)
2) Accessibility Tests Global Component Assessments Desktop Assessments Petition Petition form should be accessible.:
AssertionError: 1 accessibility violation was detected: expected 1 to equal 0
at Function.fn.(anonymous function) [as equal] (http://localhost:8000/__cypress/runner/cypress_runner.js:86181:34)
at getRet (http://localhost:8000/__cypress/runner/cypress_runner.js:89187:16)
at tryCatcher (http://localhost:8000/__cypress/runner/cypress_runner.js:139407:23)
at Function.Promise.attempt.Promise.try (http://localhost:8000/__cypress/runner/cypress_runner.js:136682:29)
at Context.thenFn (http://localhost:8000/__cypress/runner/cypress_runner.js:89201:23)
at Context.then (http://localhost:8000/__cypress/runner/cypress_runner.js:89525:21)
at Context.<anonymous> (http://localhost:8000/__cypress/runner/cypress_runner.js:100860:21)
at http://localhost:8000/__cypress/runner/cypress_runner.js:100381:33
at tryCatcher (http://localhost:8000/__cypress/runner/cypress_runner.js:139407:23)
at Promise._settlePromiseFromHandler (http://localhost:8000/__cypress/runner/cypress_runner.js:137343:31)
at Promise._settlePromise (http://localhost:8000/__cypress/runner/cypress_runner.js:137400:18)
at Promise._settlePromiseCtx (http://localhost:8000/__cypress/runner/cypress_runner.js:137437:10)
at Async../node_modules/bluebird/js/release/async.js.Async._drainQueue (http://localhost:8000/__cypress/runner/cypress_runner.js:134137:12)
at Async../node_modules/bluebird/js/release/async.js.Async._drainQueues (http://localhost:8000/__cypress/runner/cypress_runner.js:134142:10)
at Async.drainQueues (http://localhost:8000/__cypress/runner/cypress_runner.js:134016:14)
The test code is just a checka11y()
call:
[...]
context("Petition", () => {
before(() => {
cy.viewport('macbook-13');
cy.visit(`/campaigns/single-page/`);
cy.wait(500);
cy.injectAxe().configureAxe({
reporter: "v2"
});
cy.get(".sign-petition");
});
it("Petition content should be accessible.", () => {
cy.checkA11y(".sign-petition .petition-content"); // passes
});
it("Petition form should be accessible.", () => {
cy.checkA11y(".sign-petition form"); // generates those 2 errorrs
});
[...]
And I updated cypress' /plugins/index.js
to include the task logging:
module.exports = (on, config) => {
on("task", {
log(message) {
console.log(message);
return null;
},
table(message) {
console.table(message);
return null;
}
});
};
But I don't see anything in the console that lets me figure out what is actually failing and how to fix it =S
I need to disables assertions based on violations and only logs violations to a report, so was trying to pass 4 arguments to cy.checka11y() as described by @avanslaars , but got an error
0-2 arguments expected, got 4
package.json
"devDependencies": {
"@type/cypress-axe":"0.4.0"
"cypress-axe": "^0.8.1"
}
Still after reading axe-core docs cannot for the life of me figure out how to run a single rule, for example how to run just "color-contrast" rule against text on the page...
cy.checkA11y( ".example-class" )
runs the rule on whole page. I want to do a11y testing only one specific portion of the page. Do I need to pass any other configuration?
Some webpack configurations make the result of require('axe-core')
contain axe
only under a default
member variable, so window.axe
becomes undefined
here:
https://github.com/avanslaars/cypress-axe/pull/18/files#diff-168726dbe96b3ce427e7fedce31bb0bcR4
Also, @avanslaars, is there a reason you went back to window.eval(axe)
in cf6d885#diff-168726dbe96b3ce427e7fedce31bb0bcR4?
Having trouble running tests and get the following error:
cypress_runner.js:157775 TypeError: Cannot read property 'run' of undefined
at Context.<anonymous> (https://qa1.domain.com/__cypress/tests?p=cypress\support\index.js-350:18962:22)
From previous event:
at Context.thenFn (https://qa1.domain.com/__cypress/runner/cypress_runner.js:71909:23)
at Context.then (https://qa1.domain.com/__cypress/runner/cypress_runner.js:72229:21)
at Context.<anonymous> (https://qa1.domain.com/__cypress/runner/cypress_runner.js:80256:21)
at https://qa1.domain.com/__cypress/runner/cypress_runner.js:79783:33
From previous event:
at runCommand (https://qa1.domain.com/__cypress/runner/cypress_runner.js:79765:14)
at next (https://qa1.domain.com/__cypress/runner/cypress_runner.js:79900:14)
From previous event:
at next (https://qa1.domain.com/__cypress/runner/cypress_runner.js:79900:34)
From previous event:
at https://qa1.domain.com/__cypress/runner/cypress_runner.js:79937:37
From previous event:
at run (https://qa1.domain.com/__cypress/runner/cypress_runner.js:79929:15)
at Object.cy.<computed> [as percySnapshot] (https://qa1.domain.com/__cypress/runner/cypress_runner.js:80290:11)
at Context.runnable.fn (https://qa1.domain.com/__cypress/runner/cypress_runner.js:80477:20)
at callFn (https://qa1.domain.com/__cypress/runner/cypress_runner.js:30713:21)
at Test.../driver/node_modules/mocha/lib/runnable.js.Runnable.run (https://qa1.domain.com/__cypress/runner/cypress_runner.js:30706:7)
at https://qa1.domain.com/__cypress/runner/cypress_runner.js:84151:28
From previous event:
at Object.onRunnableRun (https://qa1.domain.com/__cypress/runner/cypress_runner.js:84140:17)
at $Cypress.action (https://qa1.domain.com/__cypress/runner/cypress_runner.js:78413:30)
at Test.Runnable.run (https://qa1.domain.com/__cypress/runner/cypress_runner.js:82943:20)
at Runner.../driver/node_modules/mocha/lib/runner.js.Runner.runTest (https://qa1.domain.com/__cypress/runner/cypress_runner.js:31180:10)
at https://qa1.domain.com/__cypress/runner/cypress_runner.js:31286:12
at next (https://qa1.domain.com/__cypress/runner/cypress_runner.js:31100:14)
at https://qa1.domain.com/__cypress/runner/cypress_runner.js:31110:7
at next (https://qa1.domain.com/__cypress/runner/cypress_runner.js:31042:14)
at https://qa1.domain.com/__cypress/runner/cypress_runner.js:31073:7
at next (https://qa1.domain.com/__cypress/runner/cypress_runner.js:84079:16)
at https://qa1.domain.com/__cypress/runner/cypress_runner.js:84097:11
From previous event:
at onNext (https://qa1.domain.com/__cypress/runner/cypress_runner.js:84094:57)
at done (https://qa1.domain.com/__cypress/runner/cypress_runner.js:30674:5)
at https://qa1.domain.com/__cypress/runner/cypress_runner.js:30718:11
From previous event:
at callFn (https://qa1.domain.com/__cypress/runner/cypress_runner.js:30717:10)
at Hook.../driver/node_modules/mocha/lib/runnable.js.Runnable.run (https://qa1.domain.com/__cypress/runner/cypress_runner.js:30706:7)
at https://qa1.domain.com/__cypress/runner/cypress_runner.js:84151:28
From previous event:
at Object.onRunnableRun (https://qa1.domain.com/__cypress/runner/cypress_runner.js:84140:17)
at $Cypress.action (https://qa1.domain.com/__cypress/runner/cypress_runner.js:78413:30)
at Hook.Runnable.run (https://qa1.domain.com/__cypress/runner/cypress_runner.js:82943:20)
at next (https://qa1.domain.com/__cypress/runner/cypress_runner.js:31056:10)
at https://qa1.domain.com/__cypress/runner/cypress_runner.js:31078:5
at timeslice (https://qa1.domain.com/__cypress/runner/cypress_runner.js:26146:27)
My package:
{
"devDependencies": {
"@cypress/code-coverage": "^1.8.0",
"@percy/cypress": "^2.0.0",
"axe-core": "^3.3.1",
"concurrently": "^4.1.1",
"cypress": "^3.4.1",
"cypress-axe": "^0.5.1",
"fs-extra": "^8.1.0",
"istanbul-lib-coverage": "^2.0.5",
"mocha": "^5.2.0",
"mocha-multi-reporters": "^1.1.7",
"mochawesome": "^4.1.0",
"mochawesome-merge": "^2.0.1",
"mochawesome-report-generator": "^4.0.1",
"nyc": "^14.1.1"
},
"dependencies": {
"ls": "^0.2.1",
"rimraf": "^2.6.3",
"yargs": "^13.3.0"
}
}
My test:
describe('APPB', function () {
context('[Login] Login into account', function () {
beforeEach(function(){
cy.visit('/login')
cy.injectAxe()
})
it('[Login] Redirects to /dashboard on success', function () {
cy.percySnapshot('testing before')
cy.get('input[name=username]').should('have.attr', 'placeholder', 'Username or Email address')
.type('[email protected]', { delay: 50 }).should('have.value', '[email protected]')
cy.get('input[name=password]').should('have.attr', 'placeholder', 'Password')
.type('12345678', { delay: 50 }).should('have.value', '12345678')
cy.percySnapshot('testing after')
cy.get('button.button.primary').contains('Sign in').click()
cy.checkA11y()
})
})
})
Any ideas or suggestions on what could be wrong with my setup and or test?
With Cypress 5.0 released, could this be added now to peerDependencies? I don't know if other changes would be needed.
"peerDependencies": { "cypress": "^3.1.1 || ^4.0.2" },
Thank you very much for contributing cypress-axe
to the community!
I noticed that around 0.7.0
it was decided to bundle axe-core
and I guess I'm wondering if would it might be better to keep axe-core
as a peerDependency so that consumers can exert some level of control over their version? I notice that there is already a request (#36) to upgrade the bundled version.
Reasons why I ask:
Maybe it's just a recent phenomenon, but I notice that Deque has had a few minor/patch releases to fine-tune the aggressiveness of the axe ruleset. In my application I've had to upgrade a few times (to 3.4, to 3.5) to get around "overly aggressive" (my words, not theirs) errors that were not failing in other a11y checkers.
I could be doing it wrong TM (and therefore it may not be a valuable use case), but in our application we currently also use axe-core
directly in some of our faster JSDOM tests that are not covered by our slightly slower full-browser cypress
tests. While not maybe technically necessary, it would be nice to have the a single axe-core
version/ruleset apply.
Any thoughts are appreciated. Thank you again for creating this useful library.
checkA11y()
method make it awkward to use and very difficult to extend. We have several open pull requests that are adding a new argument at the end.checkA11y()
method, and uses a custom context
argument. This makes the usage less idiomatic and doesn't allow integration with tools like Cypress Testing Library.checkA11y()
method, we have to pass things like violationCallback
in every call or create a custom command.Related issues: #40, #49, #62, #67, #68
context
argument of the checkA11y()
method, and rely on the Cypress chaining instead:cy.checkA11y() // Check the whole document
cy.getByTestId('my-modal').checkA11y() // Check part of the document
interface CypressAxeOptions {
axeOptions?: axe.RunOptions, // This used to be `options` argument
shouldFail?: (violations: RunResults[], results: RunResults[]) => boolean, // Replaces `includedImpacts` and `skipFailures`
reporters?: Reporter[]
}
type Reporter = (results: RunResults[]) => void
interface RunResults {
filename: string, // Test file
label?: string, // Label passed to checkA11y
results: axe.Result[]
}
The defaults are going to be:
{
axeOptions: undefined,
shouldFail: (violations) => violations.length > 0, // Fail on any number of violations
reporters: [
require('cypress-axe/cli-reporter')
]
}
Reporters are replacing the violationCallback
: they are more flexible, have access to both passes and violations, and you could have global reporters that would run after all tests, or pass a reporter for a separate checkA11y()
method call. The default reporter is printing an overall summary after all tests.
checkA11y()
method:cy.checkA11y(options?: CypressAxeOptions, label?: string)
Similar to labels describe()
or it()
methods, to identify results of a particular call.
cy.configureCypressAxe(options: CypressAxeOptions)
It should accept the same object as the checkA11y()
method, and set default values that could be overwritten in checkA11y()
.
I hope this covers all the use cases we have. I'm not sure that all the names are clear, so if you have any feedback, leave a comment ;-)
Hello there,
Im using cypress with typescript and when i set the variable:
const A11yConfig: Options = {
includedImpacts: ["critical"],
runOnly: {
type: "tag",
values: ["section508"]
}
};
Then, i use the cy.checkA11y(A11yConfig)
, when i run the tests an error happens:
axe.run arguments are invalid
Am i doing something wrong?
Thanks!
Versions:
"cypress": "^4.12.1"
"@types/cypress-axe": "^0.8.0"
"axe-core": "^3.5.5"
Hi,
Great project, thanks!
Just been trying to get this running with a Cypress project in TypeScript but came across some issues, so I am proposing some readme updates for typescript to state that typings are available here
https://www.npmjs.com/package/@types/cypress-axe
https://github.com/DefinitelyTyped/DefinitelyTyped/tree/master/types/cypress-axe
And that it won't work with webpack, but will work with browserify pre-processors
Related issues
Cypress Typescript users can switch from webpack, to browerify
https://github.com/cypress-io/cypress-browserify-preprocessor
Happy to make the readme changes if required ๐
Hi, is it possible to disable a rule via ID when running cy.checkA11y() on a page?
We're already aware of the color-contrast issue we have on our pages and until our site rebrand this will remain an a11y issue.
I've read the provided documentation and tried a few different variations (passing just an array through, removing [] brackets etc) of the below but the rule still appears to be checked:
cy.injectAxe()
cy.configureAxe({
rules: [{ 'aria-hidden-focus': { enabled: false } }]
})
cy.checkA11y()
Update the axe-core package to version <= 3.5.0 to include the 3.5 ruleset
Hi,
Is there a way to disable certain axe rules? We're looking to run it through CI and gradually update accessibility errors as we go then enable the rules once they're updated.
Is there a way to do this?
Thanks
I'm working on a component library so my focus in a11y testing of the components rather than the test harness they're loaded in. In protractor we can do something like:
browser.runAxeTest('Breadcrumbs', 'lib-breadcrumbs');
Looking at some of the discussion here, it seems like axe-core allows specificity.
It looks like it might be a simple passing of a selector as a parameter to the cy.checkA11y();
to propagate to axe API.
Does this seem like it would be a good approach? Or would using axe directly be preferable?
I'm not sure if anyone else if having this issue. However, when running cy.checkA11y three times, with the following params:
cy.checkA11y(null, null, null, {skipFailures: true});
it's causing the Cypress test to freeze all together with no failures
Hi, I am currently using cypress-axe
and I'm experiencing the same issues that were identified in this issue that's been closed:
#22
I left a comment there but didn't get a response and I tried implementing the other attempts on that issue board. I was just hoping to get a clear understanding of what the fix was or if there was a fix/workaround at all. Below are the versions I'm using for Cypress:
Cypress version: 6.1
cypress-axe version: 0.12.0
I am also using cypress-axe
in conjunction with Storybook: 5.3.21.
Hi Guys,
Is it possible to report/log accessibility violations only and not making our tests to fail?
Patrick
cy.readFile() must be passed a non-empty string as its 1st argument. You passed: 146.Learn more
This happens after finishing setup and trying to run a sample test.
This is in my command that I use to login:
.then((url) => {
cy.intercept('api/users/user/profile/').as('login');
cy.visit(url);
cy.wait('@login');
cy.injectAxe();
}),
Error happens on the cy.injectAxe()
line.
Thanks!
Causes eslint error "Unable to resolve path to module cypress-axe."
workaround is to use 0.8.1
Build script seems to work, but what comes from npm -i to node_modules is indeed missing the whole dist directory.
Hi All,
I am trying to setup Ally in my project but I am getting below error running an initial/simple test:
Do you know where is the issue? is due to Cypress 5? it seems that it is supported based on the documentation.Do you know which with versions runs properly?
Hi,
is it possible to configure cypress-axe, so it fails tests only if there, for example, critical
violations?
Would it be possible to update the axe-core package to the latest version in order to include the larger rule set that came in with 4.1? Thank you.
Related: #22, vercel/next.js#7945
Full repo: here (note: some of the packages there are not yet published, but the test can run without them)
I have a simple test:
import formatAxeLog from "../helpers/formatAxeLog";
describe("Home page", () => {
beforeEach(() => {
cy.visit("/");
});
it("Has no detectable a11y violations on load", () => {
cy.injectAxe();
// cy.wait(500);
cy.checkA11y(null, null, formatAxeLog);
});
});
When I run the tests multiple times they sometimes fail (due to vercel/next.js#7945, as they should), sometimes pass. The only "reliable" way is just to wait. Am I doing something wrong, or this is a bug?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.