karatelabs / karate Goto Github PK
View Code? Open in Web Editor NEWTest Automation Made Simple
Home Page: https://karatelabs.github.io/karate
License: MIT License
Test Automation Made Simple
Home Page: https://karatelabs.github.io/karate
License: MIT License
Now that we are in the same page ( I certainly appreciate it ) this might be a good time to look into this : https://arxiv.org/abs/1604.05903.
https://www.linkedin.com/pulse/testing-software-today-being-declarative-i-nabarun-mondal
If you want this framework to be awesome - you need the higher order functions.
https://en.wikipedia.org/wiki/Higher-order_function
So, here are the following functions - which are required:
And then we are good to go - almost.
Because then you need a whole library on top of set algebra.
https://en.wikipedia.org/wiki/Algebra_of_sets
From the closed issue - we only have for all function as of now.
But lack of rest will make validation sub optimal.
Good Job - 👍 - thanks again!
Hello,
I've seen that Karate supports SOAP testing, is there demo of such thing. I'd love to take a look as I am researching on tool to put in test for web services. Thank you
Karate already has the contains
keyword that can look for sub-set matches within a JSON array.
One of the good points @nmondal brought up in #1 is that there should be an easy way to apply a predicate to ALL elements in an array, and the primary use-case would be for schema-like validation.
Here is the proposal:
Given def cat =
"""
{
name: 'Billie',
rivals: [
{ id: 23, name: 'Bob' },
{ id: 42, name: 'Wild' }
]
}
"""
Then match all cat.rivals[*] == { id: '#number', name: '#string' }
Hi,
I was looking at the implementation of the Match.
Here is what I found :
https://github.com/intuit/karate/blob/master/karate-core/src/main/java/com/intuit/karate/MatchType.java
We can see that it is an enum. There is a problem with this sort of approach, that is because logical expression can be chained. Formally one can not enumerate over all operator combinations - it can not be an enum.
A formal way of doing this are by using : https://en.wikipedia.org/wiki/Predicate_(mathematical_logic)
where we do these:
for_all x in S : p(x) is true
there_exists an x in S : p(x) is true
find_one x from S : p(x) is true
But to do so, we need to have automatic variable push - and an expression evaluation. This was the reason - I started using http://commons.apache.org/proper/commons-jexl/ , but then it was, and still is - too too buggy.
With an expression evaluator - we can do awesome, or you can write your language. :-)
It's been great help till now in resolving my issues . Now am getting stuck at small issues like , If I need to parse results from a response and match a certain piece of data Say for eg :
I am using this REST API http://services.groupkt.com/country/get/all and I want to validate if my response has one particular country in the reponse how can I create that scenario.
Also if I need to capture the response in a local variable and parse and mach the information . Is it possible ?
thanks
Hi,
I am trying to use Karate with BDD for SOAP requests and try to capture Status & Response but everytime I get response as null.
I am using this URL : http://www.thomas-bayer.com/axis2/services/BLZService?wsdl
with action "getBank" and Parameter "10010424"
I am trying to use the BDD approach for calling
Given url 'http://www.thomas-bayer.com/axis2/services/BLZService/getBank'
And param blz = '10010424'
Then status 200
OR
Given url 'http://www.thomas-bayer.com/axis2/services/BLZService?wsdl'
When request 'getBank'
And param blz = '10010424'
Then status 200
Please suggest if I am mising something or we have a proper function to send this information and capture response.
Thanks
Karate does not read js files if specified in a file path. Throws a runtime exception.
For example: a/b/c/test.js cannot be read
When I try to make use of Karate testng dependency and run the tests, the following exception is seen and the tests are NOT run: Please have this issue fixed:
org.testng.TestNGException: org.xml.sax.SAXParseException; lineNumber: 3; columnNumber: 44; Attribute "parallel" with value "none" must have a value from the list "false methods tests classes instances ".
at org.testng.TestNG.initializeSuitesAndJarFile(TestNG.java:325)
at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:90)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:206)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:177)
at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:125)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
Caused by: org.xml.sax.SAXParseException; lineNumber: 3; columnNumber: 44; Attribute "parallel" with value "none" must have a value from the list "false methods tests classes instances ".
at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)
at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.error(ErrorHandlerWrapper.java:134)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:396)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:284)
at com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.validateDTDattribute(XMLDTDValidator.java:1392)
at com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.addDTDDefaultAttrsAndValidate(XMLDTDValidator.java:1311)
at com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.handleStartElement(XMLDTDValidator.java:1917)
at com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.startElement(XMLDTDValidator.java:742)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:380)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl$NSContentDriver.scanRootElementHook(XMLNSDocumentScannerImpl.java:614)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:3135)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:880)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:118)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:327)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:195)
at org.testng.xml.XMLParser.parse(XMLParser.java:39)
at org.testng.xml.SuiteXmlParser.parse(SuiteXmlParser.java:16)
at org.testng.xml.SuiteXmlParser.parse(SuiteXmlParser.java:9)
at org.testng.xml.Parser.parse(Parser.java:172)
at org.testng.TestNG.initializeSuitesAndJarFile(TestNG.java:305)
... 9 more
Do you have any option for SOAP XML request template and reusing example table
In the below example there are two place holders one for filling firstName and lastname
i will define EXAMPLE table with firstName and lastName alone but i dont define AGE in that case AGE tag should be get removed.
Sample
XML request format
<query>
<name>
<firstname>${firstName}</firstname>
<lastname>${lastName}</lastname>
</name>
<age>${age}</age>
</query>
Could you please provide solution for this scenario
match contains
is great and all but closer inspection of the REST-Assured home-page example made me realize that the need to validate that ALL elements exist but in ANY ORDER - is a valid use-case.
This should be easily implementable similar to how match each
was just implemented. Will be available in v0.1.6
Given path ‘lotto’, 5
When method get
Then status 200
And match $.lotto.lottoId == 5
And match $.lotto.winners[*].winnerId contains only [23, 54]
Hi Peter,
I am testing SOAP requests where after passing values to first request I need to extract the created employee ID and pass this to next request to verify rest of the data.
When I use my request : I get an eployeeID in my response which I collect using
Now I need to pass this same EId to next request and fetch rest of information.
What's the best way to do this. Also how can we call two reuest in same feature file when we have already defined the URL and basic authentication in Background tags in the begining. Can we reuse if there are more than 1 request in 1 scenario.
Hi,
If we look into this :
https://github.com/intuit/karate/blob/master/karate-core/src/main/java/com/intuit/karate/validator/StringValidator.java
It is obvious it is trying to do this :
object isa 'string'
But that does not solve it. What we really want is a type check operator or rather say validator.
Basic types should be, if you go ECMA way:
Now, there is a definite regex way of doing it - e.g. put back the string value of the object and then if it matches some specific regex - call it that type - there is really no other way finding what numeric type it is.
Another thing would be - replacing StringValidator with TypeCheckValidator - which takes 2 inputs:
TypeCheckValidator( object, type_id )
in this case - you can check if a type_id is known - then you can do it yourself. If it is unknown, then you can tap into the regex implementation - that is if a type_id is a regex, then
Your call.
Hello,
I would like to know if there is a way out-of-the-box to beautify JSON in the response, instead of having it 1-liner always. I know that the raw http response is logged 'as it is' but would be nice to beautify JSON, specially when testing new feature files and its endpoints.
Best,
Neill
I mean, why not !
Given def temperature = { celsius: 100, fahrenheit: 212 }
# in v0.2.1 this is supported
Then match temperature contains { fahrenheit: '#? _ == $.celsius * 1.8 + 32' }
# this is not yet supported
And match temperature == { celsius: '#number', fahrenheit: '#($.celsius * 1.8 + 32)' }
Currently the support for things like #regexSTR is limited to string values. There is no way to validate if a JSON value is present AND if it is typeof
'number'
Wondering if there would be a nice way to plug in an expression as well.
For example { month: '#? _ > 0 && _ < 13' }
Summoning @Sdaas and @redzedi what do you think of the above validation 'marker' syntax proposal ?
* def json = { foo: [{ bar: 1}, {bar: 2}, {bar: 3}]}
* def list = json.foo
* match list[0].bar == 1
Fails with:
java.lang.RuntimeException: not json, cannot do json path for value: [type: LIST, value: [{"bar":1},{"bar":2},{"bar":3}]], path: $[0].bar
at com.intuit.karate.Script.matchJsonPath(Script.java:497)
Hi Peter,
I have recently started using Karate and I am loving it so far.
Because, I have extensive experience in using Cucumber before and I feel right at home with Karate.
Kudos !!!
Although, I realised that the karate ignores the body passed with a DELETE request.
I know this is a standard practise jn the RESTful services world and it is a recommended approach.
But, the API that I am testing has made a conscious decision to not follow it based on our circumstances.
In our case we have 'relationship' resource which does not have an ID because it doesn't have a existence on it's own.
Hence in our case, we make a DELETE request to this resource with a BODY. Please find below a sample request:
Scenario: Validate body is passed with a DELETE request
Given path 'document1', 'relationships'
And request
"""
<Relationships> <Relationship TypeID="VALID" TargetDocumentId="document1"/> </Relationships>
"""
When method DELETE
Then status 200
What is your opinion about making the request BODY optional for DELETE method. Meaning pass the ENTITY for a DELETE request if it is present or else just make the request without it.
Currently, I am using patched version of Karate and Jersey Client which allows me to do this. If you are inclined towards doing it, I would be happy to share my patch.
I only recently came to know of Spring REST Docs: https://projects.spring.io/spring-restdocs/ and it is interesting.
Spring REST Docs has built-in support for REST-Assured. IMO it is a no-brainer for Spring REST Docs to support Karate. I was impetuous enough to ask the creator of Spring REST Docs @wilkinsona to consider this (via twitter), but I'm not sure I made my point well enough.
Since my priority right now is to build a community, spread the word, and ensure any remaining rough-edges are smoothened out, implementing Spring REST Docs support is not something I have bandwidth for right now. Hence the 'help wanted' tag on this ticket.
This may be a good academic exercise for validating the design of karate-core, for example if you look at the Spring REST Docs core, it has abstractions for all the HTTP things you expect, such as operations and headers.
I'm also open to the possibility of Karate implementing a clean-room documentation approach similar to Spring REST Docs. The BIG advantage would be to make this accessible to non-Java programmers. It may be simpler because we don't have to work with the abstractions of Spring REST Docs.
This works fine
Feature: Intelligence
Scenario: Creates a new model
Given url urlBase + '/model'
When method get
Then status 200
And match response == { success: true, result: '#notnull' }
But when I change the method to post
Feature: Intelligence
Scenario: Creates a new model
Given url urlBase + '/model'
When method post
Then status 200
And match response == { success: true, result: '#notnull' }
I get this NullPointerException
java.lang.NullPointerException
at com.intuit.karate.StepDefs.method(StepDefs.java:197)
at ✽.When method post(intelligence/intelligence.feature:6)
I'm able to make POST requests to the endpoint via PostMan just fine
Hi
I tried upgrading to Karate v0.3.0 and v0.3.1
When I try to run my tests using cucumber options, I get following exception:
karate_exception.txt
My runner class looks like following
`package linking;
import com.intuit.karate.junit4.Karate;
import cucumber.api.CucumberOptions;
import org.junit.runner.RunWith;
@RunWith(Karate.class)
@CucumberOptions(features =
{"src/test/java/linking/invalid_direction_id_exceptions.feature"},
format = {"pretty", "html:reports"},
tags = {"@run"})
public class Http_401_404_400_ResponsesTest {
}`
based on feedback:
I would like to be able to assert whether a HTTP request is completed within a certain amount of time.
Trying to count the records on path /response/records/record as part of validation. Given a step using xpath count function like:
And match count(/response/records/record) contains '1'
There is a NullPointerException:
java.lang.NullPointerException
at com.intuit.karate.Script.matchNamed(Script.java:436)
at com.intuit.karate.StepDefs.matchNamed(StepDefs.java:497)
at com.intuit.karate.StepDefs.matchContains(StepDefs.java:493)
at ✽.And match count(/response/records/record) contains '1'(test/uc/uctest.feature:44)
When I use the source XML from my response and use xpath evaluator at http://www.freeformatter.com/xpath-tester.html I get the expected '1' result on evaluation of the same xpath expression.
There was a rush to introduce ssl support and we ended up with the syntax ssl enabled
Come to think of it there may be a few more similar needs in the future. An immediate need that comes to mind is setting a default time-out duration.
Would be great to get feedback on the following proposals. I'd like to "unify" the means to set config within the existing keywords as far as possible. One principle is to not bloat the syntax.
set
, and Karate will introduce a config
special variable. Downside is that the name config
could tend to be used as a variable name by users. Also, extra code needs to be added to "detect" a property change* set config.ssl.enabled = true
assert
, interesting. I think I'll keep it that way and restrict the extensibility entry-point of Karate to the call
keyword* assert configure({ sslEnabled: true })
# if we decide to introduce an 'execute' keyword, but 'side-effects' are bad :P
* execute configure({ sslEnabled: true }}
# maybe one keyword for all 'configure' needs would be a good solution
* configure { sslEnabled: true }
# one nice thing is possibly multiple config in one operation
* configure { sslEnabled: true, timeOut: 60 }
configure
function* call configure { sslEnabled: true, timeOut: 60 }
Hi,
As per your last reply for my understanding for SOAP request , you recommended passing the complete SOAP request in Given Request ' <soapenv:Envelope .......>........</soapenv:Envelope >
This will be OK if my request is small , but if my request has multiple tags and few sets of data to be tested against that same tags., Is there we can parametrize our request and store this big request in some file and call it from there. As maintaining a big SOAP request in actual feature file will be confusing .
Please suggest.
thanks
Getting runtime expception when using callonce read statement. Could you please help me to resolve this issue
java.lang.RuntimeException: script failed: callonce read('classpath:com/qatooling/idr/authtoken.feature')
at com.intuit.karate.Script.evalInNashorn(Script.java:303)
at com.intuit.karate.Script.evalInNashorn(Script.java:277)
at com.intuit.karate.Script.eval(Script.java:224)
at com.intuit.karate.Script.assign(Script.java:453)
at com.intuit.karate.Script.assign(Script.java:419)
at com.intuit.karate.StepDefs.def(StepDefs.java:149)
at ✽.* def token = callonce read('classpath:com/qatooling/idr/authtoken.feature')(com/qatooling/idr/idr.feature:8)
Caused by: javax.script.ScriptException: :1:9 Expected ; but found read
callonce read('classpath:com/qatooling/idr/authtoken.feature')
^ in at line number 1 at column number 9
at jdk.nashorn.api.scripting.NashornScriptEngine.throwAsScriptException(NashornScriptEngine.java:467)
at jdk.nashorn.api.scripting.NashornScriptEngine.compileImpl(NashornScriptEngine.java:534)
at jdk.nashorn.api.scripting.NashornScriptEngine.compileImpl(NashornScriptEngine.java:521)
at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:399)
at jdk.nashorn.api.scripting.NashornScriptEngine.eval(NashornScriptEngine.java:155)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)
at com.intuit.karate.Script.evalInNashorn(Script.java:298)
at com.intuit.karate.Script.evalInNashorn(Script.java:277)
at com.intuit.karate.Script.eval(Script.java:224)
at com.intuit.karate.Script.assign(Script.java:453)
at com.intuit.karate.Script.assign(Script.java:419)
at com.intuit.karate.StepDefs.def(StepDefs.java:149)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at cucumber.runtime.Utils$1.call(Utils.java:40)
at cucumber.runtime.Timeout.timeout(Timeout.java:16)
at cucumber.runtime.Utils.invoke(Utils.java:34)
at cucumber.runtime.java.JavaStepDefinition.execute(JavaStepDefinition.java:38)
at cucumber.runtime.StepDefinitionMatch.runStep(StepDefinitionMatch.java:37)
at cucumber.runtime.Runtime.runStep(Runtime.java:300)
at cucumber.runtime.model.StepContainer.runStep(StepContainer.java:44)
at cucumber.runtime.model.StepContainer.runSteps(StepContainer.java:39)
at cucumber.runtime.model.CucumberScenario.runBackground(CucumberScenario.java:59)
at cucumber.runtime.model.CucumberScenario.run(CucumberScenario.java:42)
at cucumber.runtime.junit.ExecutionUnitRunner.run(ExecutionUnitRunner.java:102)
at cucumber.runtime.junit.FeatureRunner.runChild(FeatureRunner.java:63)
at cucumber.runtime.junit.FeatureRunner.runChild(FeatureRunner.java:18)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at cucumber.runtime.junit.FeatureRunner.run(FeatureRunner.java:70)
at com.intuit.karate.junit4.Karate.runChild(Karate.java:60)
at com.intuit.karate.junit4.Karate.runChild(Karate.java:25)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at com.intuit.karate.junit4.Karate.run(Karate.java:66)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: jdk.nashorn.internal.runtime.ParserException: :1:9 Expected ; but found read
callonce read('classpath:com/qatooling/idr/authtoken.feature')
^
at jdk.nashorn.internal.parser.AbstractParser.error(AbstractParser.java:292)
at jdk.nashorn.internal.parser.AbstractParser.error(AbstractParser.java:277)
at jdk.nashorn.internal.parser.AbstractParser.expectDontAdvance(AbstractParser.java:348)
at jdk.nashorn.internal.parser.AbstractParser.expect(AbstractParser.java:335)
at jdk.nashorn.internal.parser.Parser.endOfLine(Parser.java:3369)
at jdk.nashorn.internal.parser.Parser.expressionStatement(Parser.java:1162)
at jdk.nashorn.internal.parser.Parser.statement(Parser.java:969)
at jdk.nashorn.internal.parser.Parser.sourceElements(Parser.java:775)
at jdk.nashorn.internal.parser.Parser.program(Parser.java:711)
at jdk.nashorn.internal.parser.Parser.parse(Parser.java:284)
at jdk.nashorn.internal.parser.Parser.parse(Parser.java:250)
at jdk.nashorn.internal.runtime.Context.compile(Context.java:1281)
at jdk.nashorn.internal.runtime.Context.compileScript(Context.java:1248)
at jdk.nashorn.internal.runtime.Context.compileScript(Context.java:629)
at jdk.nashorn.api.scripting.NashornScriptEngine.compileImpl(NashornScriptEngine.java:532)
... 49 more
java.lang.RuntimeException: script failed: callonce read('classpath:com/qatooling/idr/authtoken.feature')
at com.intuit.karate.Script.evalInNashorn(Script.java:303)
at com.intuit.karate.Script.evalInNashorn(Script.java:277)
at com.intuit.karate.Script.eval(Script.java:224)
at com.intuit.karate.Script.assign(Script.java:453)
at com.intuit.karate.Script.assign(Script.java:419)
at com.intuit.karate.StepDefs.def(StepDefs.java:149)
at ✽.* def token = callonce read('classpath:com/qatooling/idr/authtoken.feature')(com/qatooling/idr/idr.feature:8)
Caused by: javax.script.ScriptException: :1:9 Expected ; but found read
callonce read('classpath:com/qatooling/idr/authtoken.feature')
^ in at line number 1 at column number 9
at jdk.nashorn.api.scripting.NashornScriptEngine.throwAsScriptException(NashornScriptEngine.java:467)
at jdk.nashorn.api.scripting.NashornScriptEngine.compileImpl(NashornScriptEngine.java:534)
at jdk.nashorn.api.scripting.NashornScriptEngine.compileImpl(NashornScriptEngine.java:521)
at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:399)
at jdk.nashorn.api.scripting.NashornScriptEngine.eval(NashornScriptEngine.java:155)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)
at com.intuit.karate.Script.evalInNashorn(Script.java:298)
at com.intuit.karate.Script.evalInNashorn(Script.java:277)
at com.intuit.karate.Script.eval(Script.java:224)
at com.intuit.karate.Script.assign(Script.java:453)
at com.intuit.karate.Script.assign(Script.java:419)
at com.intuit.karate.StepDefs.def(StepDefs.java:149)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at cucumber.runtime.Utils$1.call(Utils.java:40)
at cucumber.runtime.Timeout.timeout(Timeout.java:16)
at cucumber.runtime.Utils.invoke(Utils.java:34)
at cucumber.runtime.java.JavaStepDefinition.execute(JavaStepDefinition.java:38)
at cucumber.runtime.StepDefinitionMatch.runStep(StepDefinitionMatch.java:37)
at cucumber.runtime.Runtime.runStep(Runtime.java:300)
at cucumber.runtime.model.StepContainer.runStep(StepContainer.java:44)
at cucumber.runtime.model.StepContainer.runSteps(StepContainer.java:39)
at cucumber.runtime.model.CucumberScenario.runBackground(CucumberScenario.java:59)
at cucumber.runtime.model.CucumberScenario.run(CucumberScenario.java:42)
at cucumber.runtime.junit.ExecutionUnitRunner.run(ExecutionUnitRunner.java:102)
at cucumber.runtime.junit.FeatureRunner.runChild(FeatureRunner.java:63)
at cucumber.runtime.junit.FeatureRunner.runChild(FeatureRunner.java:18)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at cucumber.runtime.junit.FeatureRunner.run(FeatureRunner.java:70)
at com.intuit.karate.junit4.Karate.runChild(Karate.java:60)
at com.intuit.karate.junit4.Karate.runChild(Karate.java:25)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at com.intuit.karate.junit4.Karate.run(Karate.java:66)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: jdk.nashorn.internal.runtime.ParserException: :1:9 Expected ; but found read
callonce read('classpath:com/qatooling/idr/authtoken.feature')
^
at jdk.nashorn.internal.parser.AbstractParser.error(AbstractParser.java:292)
at jdk.nashorn.internal.parser.AbstractParser.error(AbstractParser.java:277)
at jdk.nashorn.internal.parser.AbstractParser.expectDontAdvance(AbstractParser.java:348)
at jdk.nashorn.internal.parser.AbstractParser.expect(AbstractParser.java:335)
at jdk.nashorn.internal.parser.Parser.endOfLine(Parser.java:3369)
at jdk.nashorn.internal.parser.Parser.expressionStatement(Parser.java:1162)
at jdk.nashorn.internal.parser.Parser.statement(Parser.java:969)
at jdk.nashorn.internal.parser.Parser.sourceElements(Parser.java:775)
at jdk.nashorn.internal.parser.Parser.program(Parser.java:711)
at jdk.nashorn.internal.parser.Parser.parse(Parser.java:284)
at jdk.nashorn.internal.parser.Parser.parse(Parser.java:250)
at jdk.nashorn.internal.runtime.Context.compile(Context.java:1281)
at jdk.nashorn.internal.runtime.Context.compileScript(Context.java:1248)
at jdk.nashorn.internal.runtime.Context.compileScript(Context.java:629)
at jdk.nashorn.api.scripting.NashornScriptEngine.compileImpl(NashornScriptEngine.java:532)
... 49 more
Reporting needs to be enhanced when a user accidentally uses a keyword as variable. Currently the exception generated by karate does not give this info.
There should be a docker setup to quickstart the testing. Any developer/tester just have to pull the docker, mount the Karate scripts and run the tests. This approach will help to run tests on clusters, hence finishing the tests faster.
I would like to contribute this feature.
Hi Guys,
here is what I found:
https://github.com/intuit/karate/blob/master/karate-core/src/main/java/com/intuit/karate/validator/ValidationResult.java
Formally, from a validation perspective - if you choose to do so, you need to have these:
Object expected
Object actual
String message
String expression
Let's look at them. A validation is formally :
Validate(expected,actual,validation_function)
// which internally does this :
if ( validation_function(expected,atual) ) {
// do pass
} else {
// do fail
}
Now the problem is - you can not pass the validation_function - hence the expression.
Also logging expected and actual will help tremendously. This is not a blocking issue, it is more on the line of enrichment. Please think about it.
Hi,
I am setting default value for Authorization header in Background so that I don't end up repeating it in every Scenario.
But I also had a Scenario in which I need a different value for Authorization header. I was setting it again in the Scenario and it used to overwrite the previous value. This worked great until the recent version v0.3.1 of Karate. In the latest Karate version for some reason it keeps both values (first_value, second_value).
This results in Authorization error.
Feature: Sample
Background:
url someServiceUrl
header Authorization = "Basic Token_1"
Scenario: Override the Authorization header in background
Given path some_path
And header Authorization = "Basic Token_2"
This results in a request which has Authorization values as
"Basic Token_1, Basic Token_2"
Hi,
I have been using REST assured for past 2 years. When I see karate my first impression is wow. I'm in the process of trying to incorporate karate in my automation suites.
I have requirement to run API test first and then to ensure such changes reflected in web application. In such case how to integrate?
Currently, I'm using Cucumber BDD - Java, Selenium +Rest Assured. I achieved above integration test by hitting rest call and verifying in UI. In such scenario, I write code in step definition for REST and UI.
Could you please let me know? is it possible to implement such scenario with Karate?
This is a help-wanted ticket. Looks like there is a need to add a section on API testing tools at this GitHub project which is a guide / resource-list on test-automation frameworks.
Here is a link to that projects issue tracker with more details:
Speaking of Cucumber Data Tables.
Every time I try to the call-feature demo or use similar setup on my project I get an error that file 'create-two-cats.feature' does not exist.
Do I need to make any changes before running call-feature demo file?
Although Karate includes data-driven testing and tags, it would be nice to support TestNG.
Currently Karate has a hard-dependency on JUnit but the role of JUnit is simply a test-runner, nothing else.
The main reason why a TestNG option would be good to have is to allow teams already using TestNG to introduce Karate side-by-side with existing Java-based web-API tests.
Details here: skyscreamer/JSONassert#44
This will be easy, the only thing the org.json artifact is doing at the moment is XML --> JSON conversion.
Feature: Rest API Demo Showing use of Data table using Karate Framework
Background:
* url 'http://services.groupkt.com/country/get/iso2code'
Scenario: Verify the API is up and running and returning the details for each country from data table
Examples:
|name |
|CA |
|IN |
|US |
Given path ''
When method get
And status 200
In this case have tried various options but not sure how to use these countrycodes one by one and capture the response. Can you please suggest how to use data in this case.
Thanks
Sonia
Good to have support similar to "soft assertions" in testng. This will help to get all the list of failures in one shot instead of running the tests multiple times
Writing reusable functions it would be nice not to be limited to using JSON here.
The documentation states that
TODO: right now only JSON HTTP calls are supported by karate.request(req)
Maybe I could have a look at this .. I guess a good starting point is ScriptContext/ScriptBridge
Running a simple wiremock standalone server
java -jar wiremock-standalone-2.5.1.jar --port=9000 --verbose
The server is available via http://localhost:9000 and can be accessed via postman/curl etc
When attempting to run a test from Karate using the url above, I get the following error
Scenario: Service calls product service to get a SPORTS OFFERING Time elapsed: 0.004 sec <<< ERROR!
javax.ws.rs.ProcessingException: java.net.SocketException: Unexpected end of file from server
at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:287)
at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:252)
at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:684)
at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:681)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.process(Errors.java:228)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444)
at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:681)
at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:411)
at com.intuit.karate.StepDefs.method(StepDefs.java:318)
at ✽.When method get(services/product/offerings.feature:11)
Caused by: java.net.SocketException: Unexpected end of file from server
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:792)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:789)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:399)
at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:285)
at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:252)
at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:684)
at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:681)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.process(Errors.java:228)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444)
at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:681)
at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:411)
at com.intuit.karate.StepDefs.method(StepDefs.java:318)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at cucumber.runtime.Utils$1.call(Utils.java:40)
at cucumber.runtime.Timeout.timeout(Timeout.java:16)
at cucumber.runtime.Utils.invoke(Utils.java:34)
at cucumber.runtime.java.JavaStepDefinition.execute(JavaStepDefinition.java:38)
at cucumber.runtime.StepDefinitionMatch.runStep(StepDefinitionMatch.java:37)
at cucumber.runtime.Runtime.runStep(Runtime.java:300)
at cucumber.runtime.model.StepContainer.runStep(StepContainer.java:44)
at cucumber.runtime.model.StepContainer.runSteps(StepContainer.java:39)
at cucumber.runtime.model.CucumberScenario.run(CucumberScenario.java:44)
at cucumber.runtime.junit.ExecutionUnitRunner.run(ExecutionUnitRunner.java:102)
at cucumber.runtime.junit.FeatureRunner.runChild(FeatureRunner.java:63)
at cucumber.runtime.junit.FeatureRunner.runChild(FeatureRunner.java:18)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at cucumber.runtime.junit.FeatureRunner.run(FeatureRunner.java:70)
at com.intuit.karate.junit4.Karate.runChild(Karate.java:60)
at com.intuit.karate.junit4.Karate.runChild(Karate.java:25)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at com.intuit.karate.junit4.Karate.run(Karate.java:66)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
We are able to run against a version of the wiremock server hosted in CI via TeamCity with no issues.
Any ideas?
Thanks in advance :)
Hey, trying to send a patch request but failing.
Scenario looks like below
someUrl is defined in properties file as 'http://localhost:4000' and the service is running locally.
Scenario: Test patch method returns an error as it has no params
Given url someUrl
And path '/'
And request '{}'
When method patch
Then status 422
Logs
javax.ws.rs.ProcessingException: java.net.ProtocolException: Invalid HTTP method: PATCH
at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:287)
at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:252)
at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:684)
at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:681)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.process(Errors.java:228)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444)
at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:681)
at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:437)
at com.intuit.karate.StepDefs.method(StepDefs.java:313)
at ✽.When method PATCH(/Users/mna07/dev/<proj_name>/src/test/java/serviceTests/basket/basketStubTester.feature:20)
Caused by: java.net.ProtocolException: Invalid HTTP method: PATCH
at java.net.HttpURLConnection.setRequestMethod(HttpURLConnection.java:440)
at sun.net.www.protocol.http.HttpURLConnection.setRequestMethod(HttpURLConnection.java:552)
at org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:347)
at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:285)
at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:252)
at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:684)
at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:681)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
at org.glassfish.jersey.internal.Errors.process(Errors.java:228)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444)
at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:681)
at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:437)
at com.intuit.karate.StepDefs.method(StepDefs.java:313)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at cucumber.runtime.Utils$1.call(Utils.java:40)
at cucumber.runtime.Timeout.timeout(Timeout.java:16)
at cucumber.runtime.Utils.invoke(Utils.java:34)
at cucumber.runtime.java.JavaStepDefinition.execute(JavaStepDefinition.java:38)
at cucumber.runtime.StepDefinitionMatch.runStep(StepDefinitionMatch.java:37)
at cucumber.runtime.Runtime.runStep(Runtime.java:300)
at cucumber.runtime.model.StepContainer.runStep(StepContainer.java:44)
at cucumber.runtime.model.StepContainer.runSteps(StepContainer.java:39)
at cucumber.runtime.model.CucumberScenario.run(CucumberScenario.java:44)
at cucumber.runtime.model.CucumberFeature.run(CucumberFeature.java:165)
at cucumber.runtime.Runtime.run(Runtime.java:122)
at cucumber.api.cli.Main.run(Main.java:36)
at cucumber.api.cli.Main.main(Main.java:18)
It looks like there is no listed PATCH method in the below file
/Users/mna07/.m2/repository/org/glassfish/jersey/core/jersey-client/2.25/jersey-client-2.25-sources.jar!/org/glassfish/jersey/client/JerseyInvocation.java
and after some searching, its not supported :(
https://stackoverflow.com/questions/17897171/how-to-have-a-patch-annotation-for-jax-rs
Sorry if this has been asked before!
Ola,
Is it possible to pass in multiple param refs under a single param,
EG
I can pass in values such as
And param references = 'REF1'
And param references = 'REF2'
Which will result in the URL
<base_url>/path?references=REF1&references=REF2
Our API also accepts
<base_url>/path?references=REF1,REF2
but I cannot find a way to get param to work with multiple refs with karate
tried
And param references = ['REF1','REF2']
And param references = 'REF1','REF2'
And param references = 'REF1' + \, + 'REF2'
In lieu of above, I also attempted to modify the path directly
Given path 'products/prices?references=REF1,REF2'
Is this just unsupported in the current version of Karate, or was it purposely missed?
Thanks!
Should be simple, using the new configure
syntax. Surely should be available in 0.2.1
Hi Peter!
I am having a quite nice experience with Karate so far, thanks for the continued effort! :)
I am working with a feature with the following design:
Background:
* def xyz = callonce read('classpath:abc.feature')
* configure headers = read('headers.js')
Scenario: Does something
Scenario: Does something else
Let's say abc.feature it is a feature that authenticates my user and sets headers accordingly. From the execution logs, I am seeing:
...
cached callonce: read('classpath:abc.feature')
...
But the abc.feature is being invoked continuously, before each and every Scenario. From the 'callonce' standpoint, I guess it is not correct, this is causing me to hit the authentication endpoint unnecessarily. I am using karate 0.3.1.
Best,
Neill
Failure highlighting needs to be improved for json response having multi levels in response
Hi am working on a SOAP API , where it needs Basic Authorization ( Username , Password ) to return response. I am tryig to pass
Background:
OR
Inside steps
And header Authorization = {Username: '', Password: ''}
It's giving me error in either case.
Please suggest how can we use Basic Authorisation in our feature file.
Creating new project from archetype creates example test class with "junit" dependency. Should be "junit4"
Here is the file :
Feature: data to next api by making rest calls
Scenario: read over tons of data to process and feed it next
Given url 'https://jsonplaceholder.typicode.com/posts'
When method get
Then status 200
# here I want to verify that
# 1. The response is a non empty list.
# 2. The response contains json structures where
# all json structures are
# of the schema - { "userId": int , "id": int , "title": string , "body" : string }
# And then, we want to isolate text from body and title
# Pass it to next API call after filtering
# Then do some more business processing
# Yes, we can call any other language, but then we want less code.
# How do I do it?
# Looks to us that there is a design flaw which made Karate not Turing Complete
# More over, there is no fixed iteration like SQL
# Let me know if that understanding is correct.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.