brianmacintosh / alexa-skill-test-framework Goto Github PK
View Code? Open in Web Editor NEWFramework for easy offline black-box testing of Alexa skills.
License: MIT License
Framework for easy offline black-box testing of Alexa skills.
License: MIT License
When using "it" in stead of "describe", test case always succed.
It would be really cool to assert certain words or phrases inside the output and reprompt speech instead of matching the whole string.
With a recent update, slots now support more than just a value but a match to a category, called resolutions.
https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/custom-standard-request-types-reference#intentrequest
Would be nice to specify these parameters in the IntentRequest builder as well.
resolutions.forEach(resolution => {
^
TypeError: resolutions.forEach is not a function
at Object.addEntityResolutionsToRequest (C:\Users\Mohd Ashif Ali\Documents\GitHub\alexa-greet-my-friend\test\node_modules\alexa-skill-test-framework\index.js:357:15)
at Suite. (C:\Users\Mohd Ashif Ali\Documents\GitHub\alexa-greet-my-friend\test\greeter-test.js:36:23)
at Object.create (C:\Users\Mohd Ashif Ali\AppData\Roaming\npm\node_modules\mocha\lib\interfaces\common.js:112:19)
at context.describe.context.context (C:\Users\Mohd Ashif Ali\AppData\Roaming\npm\node_modules\mocha\lib\interfaces\bdd.js:40:27)
at Suite. (C:\Users\Mohd Ashif Ali\Documents\GitHub\alexa-greet-my-friend\test\greeter-test.js:30:2)
at Object.create (C:\Users\Mohd Ashif Ali\AppData\Roaming\npm\node_modules\mocha\lib\interfaces\common.js:112:19)
at context.describe.context.context (C:\Users\Mohd Ashif Ali\AppData\Roaming\npm\node_modules\mocha\lib\interfaces\bdd.js:40:27)
at Object. (C:\Users\Mohd Ashif Ali\Documents\GitHub\alexa-greet-my-friend\test\greeter-test.js:18:1)
at Module._compile (module.js:653:30)
at Object.Module._extensions..js (module.js:664:10)
at Module.load (module.js:566:32)
at tryModuleLoad (module.js:506:12)
at Function.Module._load (module.js:498:3)
at Module.require (module.js:597:17)
at require (internal/module.js:11:18)
at C:\Users\Mohd Ashif Ali\AppData\Roaming\npm\node_modules\mocha\lib\mocha.js:250:27
at Array.forEach ()
at Mocha.loadFiles (C:\Users\Mohd Ashif Ali\AppData\Roaming\npm\node_modules\mocha\lib\mocha.js:247:14)
at Mocha.run (C:\Users\Mohd Ashif Ali\AppData\Roaming\npm\node_modules\mocha\lib\mocha.js:576:10)
at Object. (C:\Users\Mohd Ashif Ali\AppData\Roaming\npm\node_modules\mocha\bin_mocha:637:18)
at Module._compile (module.js:653:30)
at Object.Module._extensions..js (module.js:664:10)
at Module.load (module.js:566:32)
at tryModuleLoad (module.js:506:12)
at Function.Module._load (module.js:498:3)
at Function.Module.runMain (module.js:694:10)
at startup (bootstrap_node.js:204:16)
at bootstrap_node.js:625:3
Forgive me if I'm wrong, but I don't see a way to deal with different states.
I'm talking about this:
Alexa.CreateStateHandler(States.READY, { // handlers here })
'npm install alexa-skill-test-framework --save-dev' point to this branch:
https://github.com/BrianMacIntosh/alexa-skill-test-framework/blob/v1.1.1/index.js
Which doesn't contain the latest methods on master branch. Wonder if you can bump the version? Thxs
Referrs to:
alexaTest.getIntentRequest(intentName, [slots], [locale])
AMAZON.DATE returns "undefined" (contrary to other slot types returning "?"), if no or wrong slot value has been provided in the input. There is no way without hacks to simulate this behavior in the test framework.
I'm currently using version 1.2.4. It seems that when I try to set an existing session attribute using withSessionAttributes, it doesn't override it. If the session attribute is not an existing one then it gets properly set. I downloaded the latest changes from your master and this issue doesn't happen anymore, could you please release a new version?
The Session object of every request
always has the new
attribute set to true
. This is a hardcoded value in https://github.com/BrianMacIntosh/alexa-skill-test-framework/blob/master/index.js#L400
Expected behavior would be to only have this be true for the first request in the sequence. The reference in https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/alexa-skills-kit-interface-reference states that new
"Returns true for a new session or false for an existing session."
Hello everyone, wanted to know what to pass for index for
alexaTest.initialize(index,
appId, userId, deviceId)`
I am using my custom webhook using NodeJS.
app.js -> router using alexa-verifer-middleware
app.use('/alexa', require('./routes/alexa'));
routes/alexa
router.post('/podcast', function(req, res) { const handlers = require('../handlers/alexaHandler'); if (!skill) { skill = Alexa.SkillBuilders.custom() .addRequestHandlers( handlers.LaunchRequestHandler, handlers.HelpHandler..... ).create();
handler/alexaHandler -> where all the functions are maintained like LaunchRequestHandlers and HelpIntent.
Please advice. Code is working fine but as it is becoming more complex planning to use a testing framework.
Really like the alexa-skill-test-framework, i'd love to use it with our framework https://github.com/mediarain/voxa, which seems like we could right now as it is. However, there are some assertions we'd like to do, like making sure a transition results in a change to X state, that functionality looks like it would belong inside a plugin but the current mechanism in setExtraFeature is not extensible.
Would this be something you're interested in?
This is a request for an enhancement.
I wrote a skill. Some of the skill's handlers modify attributes in the session
object. Is there a way for me to test correctness of the session
object?
I'd like to be able to test synonym resolution logic in my Alexa skill but currently this framework doesn't allow any extension of the basic {name: "SlotName", value: "SlotValue"}
structure.
I need to be able to add a resolutions object to a slot as documented here: https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/entity-resolution-for-slot-types#intentrequest-changes
See below example JSON from the above link.
"resolutions": {
"resolutionsPerAuthority": [
{
"authority": "amzn1.er-authority.echo-sdk.<skill_id>.MEDIA_TYPE",
"status": {
"code": "ER_SUCCESS_MATCH"
},
"values": [
{
"value": {
"name": "song",
"id": "SONG"
}
}
]
}
]
}
Currently getIntentRequest
does the following logic for slots:
if (!slots) {
slots = {};
}
else {
for (var key in slots) {
slots[key] = {name: key, value: slots[key]};
}
}
I would propose that instead we're allowed to inject the slots object as required. The Alexa SDKs will be changing quite frequently due to the constant shift in this new technology and frameworks like these will need to be able to keep up. So I would propose the following change:
intent": {"name": intentName, "slots": slots || {}}
And remove the if/else logic referenced above.
This allows users of the test framework to pass through whatever slot configurations they need. A user may pass through an invalid object but if they do their tests will fail and alert them to that.
I'm more than happy to raise a PR for this if the idea is accepted. We would really like to keep using this test framework.
I really like this framework. One feature I'm missing is accessToken (account linking) and permissions. Any plans on adding them?
I am trying to keep session open after response so that user does not have to say "ask to " trigger next/new intent.
Currently response ends in period "." not in question mark "?". Is "?" necessary to end the response and keep session open.
Allow switching off/on debugging tests by printing the response. Something based on
... console.log(response); var actualSay = response.response && response.response.outputSpeech ? response.response.outputSpeech.ssml : undefined; var actualReprompt = response.response && response.response.reprompt && response.response.reprompt.outputSpeech ? response.response.reprompt.outputSpeech.ssml : undefined; ...
in ../node_modules/alexa-skill-test-framework/index.js
While creating our Alexa skill, our team noticed that, if we forget to emit :responseReady
when starting to play something in the audio player, the Alexa device will call a the Unhandled
handler with the following request (see docs):
{ type: "System.ExceptionEncountered",
requestId: "amzn1.echo-api.request.170b9121-04e5-4794-826d-1c00f51dcc8c",
timestamp: "2017-10-02T11:01:02Z",
locale: "en-GB",
error:
{ type: "INVALID_RESPONSE",
message: "An exception occurred while dispatching the request to the skill." },
cause: { requestId: "amzn1.echo-api.request.cf8066e8-c7d8-4324-be51-f3c0d1ff9709" } }
So we changed our Unhandled
handler to look a bit like this:
Unhandled() {
if (this.event.request.type === 'System.ExceptionEncountered') {
// Log the system exception...
}
this.emit(':tell', 'Sorry, there was an error.');
}
We would like to test how we handle system exceptions in our skill. However, as far as I know, there is no way within the testing framework to simulate an error in the request.
The code style of the index.js differs from the JavaScript standard but is not reflected in the editorconfig. So the automatic reformat of IDE changes the whole file.
I think there are two options:
Any comments on this topic?
Some intents require attributes from the session, so it would be useful to be able to set this before running a test.
There is currently withStoredAttributes
that lets you set persistent attributes, but nothing for the session.
It is strange. For example intent 'AMAZON.StopIntent' pass the test but every test I am expecting that alexa repeat the question fails :(
HelloWorldIntent:
module.exports = {
'canHandle': (handlerInput) => {
return handlerInput.requestEnvelope.request.type === 'IntentRequest'
&& handlerInput.requestEnvelope.request.intent.name === 'HelloWorldIntent';
},
'handle': (handlerInput) => {
const requestAttributes = handlerInput.attributesManager.getRequestAttributes();
const speakOutput = requestAttributes.t(('WELCOME'));
const speakRepeat = requestAttributes.t(('REPEAT'));
return handlerInput.responseBuilder
.speak(speakOutput)
.reprompt(speakRepeat)
.getResponse();
},
};
test:
alexaTest.initialize(
require('../awsAdapter'),
'amzn1.ask.skill.00000000-0000-0000-0000-000000000000',
'amzn1.ask.account.VOID');
const textResources = require('../localization/translations.js');
alexaTest.initializeI18N(textResources);
const supportedLocales = ['en-US', 'de-DE'];
for (let i = 0; i < supportedLocales.length; i++) {
const locale = supportedLocales[i];
alexaTest.setLocale(locale);
describe('skill tests', function() {
// tests the behavior of the skill's HelloWorldIntent
describe('CancelAndStopIntentIntent', () => {
alexaTest.test([
{
request: alexaTest.getIntentRequest('AMAZON.CancelIntent'),
says: alexaTest.t('STOP_ANSWER'), shouldEndSession: true,
},
]);
});
describe('HelloWorldI ntent', function() {
alexaTest.test([
{
request: alexaTest.getIntentRequest('HelloWorldIntent'),
says: alexaTest.t('WELCOME'),
reprompt: alexaTest.t('REPEAT'),
repromptsNothing: false, shouldEndSession: false,
},
]);
});
});
};
Output:
1)
skill tests
HelloWorldI ntent
returns the correct responses:
AssertionError: Request #1 (HelloWorldIntent): Possible Certification Problem: The response keeps the session open but does not contain a question mark.
at Object._assert (/home/alexa/Templates/newAlexa/node_modules/alexa-skill-test-framework/index.js:845:9)
at CallbackContext.assert (/home/alexa/Templates/newAlexa/node_modules/alexa-skill-test-framework/index.js:39:17)
at CallbackContext._questionMarkCheck (/home/alexa/Templates/newAlexa/node_modules/alexa-skill-test-framework/index.js:64:8)
at ctx.Promise.then.response (/home/alexa/Templates/newAlexa/node_modules/alexa-skill-test-framework/index.js:728:17)
at <anonymous>
Could somebody help me please?
I am a noob in coding Alexa skills and I bump into your test framework. Cool tool btw.
I have a problem here when I try to run my tests. When I execute npm test
it doesn't terminate
I just followed the code in one of your examples as an experiment but I still have the same behavior.
const alexaTest = require('alexa-skill-test-framework');
alexaTest.initialize(
require('../index.js'),
"skill-id",
'user-id');
describe("Alexa skill using Staging configuration", function() {
describe("LaunchRequest", function() {
alexaTest.test([
{
request: alexaTest.getLaunchRequest(),
says: "Welcome!", repromptsNothing: true, shouldEndSession: true
}
]);
});
describe("CountIntent", function() {
const slot = {'companyMetadata': 'employees', 'siteName': 'fort'};
const requestWithSlot = alexaTest.getIntentRequest('CountIntent', slot);
alexaTest.test([
{
request: requestWithSlot,
says: "fort has 61 employees", shouldEndSession: true, repromptsNothing: true
}
]);
});
});
My node version is v9.0.0
package.json is
"dependencies": {
"alexa-sdk": "^1.0.12",
"dotenv": "^4.0.0",
"pg": "^7.4.0"
},
"devDependencies": {
"alexa-skill-test-framework": "^1.1.2",
"chai": "^4.1.2",
"mocha": "^4.0.1"
}
Appreciate the help.
I'm running into issues where the mock context times out after 3 seconds (which is the default). It would be great to be able to pass in options (such as timeout) to the aws-lambda-mock-context dependency. https://www.npmjs.com/package/aws-lambda-mock-context
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.