sugarcrm / thorn Goto Github PK
View Code? Open in Web Editor NEWREST API testing utilities for SugarCRM
Home Page: https://sugarcrm.github.io/thorn/
License: Apache License 2.0
REST API testing utilities for SugarCRM
Home Page: https://sugarcrm.github.io/thorn/
License: Apache License 2.0
The mobile team needs to write a thorn test to verify the /metadata endpoint. For that, the test setup needs to login as a mobile user. Add ability to specify the platform parameter for the test set-up proc.
Agent.get
, Agent.post
, etc. should throw an error if they detect an OAuth-Token header.
This is something a number of developers have gotten confused about; throwing will make it clearer that it's not supported.
In general, I'd like to make Thorn's error messages more self-documenting and helpful to the person writing a Thorn test; Thorn should help them debug rather than emit information only comprehensible to Thorn's maintainers. Thorn throwing an error doesn't, and shouldn't, imply there's a bug in Thorn and I'd like to make that more clear, for everyone's benefit.
Of course, we have to make sure that this doesn't interfere with Thorn's internal login/refresh code.
While not a perfect fix, this would at least make it line up slightly better with Mocha's spec
reporter.
As part of the refactor done in #212 a decision was made towards having x-thorn
header matching the username
of the agent triggering a certain action, when this action is triggered by Fixtures, making it behave pretty much the same way like everywhere else when Agents trigger their own actions.
In order to provide additional information while debugging we want to introduce an additional x-thorn header where context is provided, e.g: X-Thorn-Context: Fixtures
or X-Thorn-Context: MetadataFetcher
yarn test -- --coverage
yarn run v1.3.2
warning From Yarn 1.0 onwards, scripts don't require "--" for options to be forwarded. In a future version, any explicit "--" will be forwarded as-is to the scripts.
It should be possible to perform several lookups at once with a single call to Fixtures.lookup
; in particular, I think something like the following should work:
let [record1, record2, record3] = Fixtures.lookup('MyModule', [
{
name: 'record 1',
},
{
name: 'record 2',
},
{
name: 'record3',
}
]);
I think it's possible to do this in a backwards-compatible way too, similar to how we handle it for Fixtures.create
. Nor do I think it should be particularly difficult, as this is a totally synchronous and promise-free function.
Have to wait until sugarcrm/javascript#4 is resolved
https://github.com/sugarcrm/thorn/pull/129/files#r100950952
alias-mac
when we move to travis ci, can we also move towards adding scripts directly in the package.json instead of using gulp? Many ppl moved towards that and imho that is the right way to go, since doing gulp test or npm test is exactly the same from the dev perspective and saves us from having to install/depend on another tool.
This should help clear up some misconceptions about what Thorn does and what you should use it for.
Low priority.
Previously, we allowed caching of metadata in a file. This greatly sped up development time, since the metadata fetch is very time consuming.
We already cache the metadata between tests. This is to cache the metadata between test runs, perhaps in a .json file somewhere.
57 passing (6s)
12 failing
1) Thorn Fixtures creation should create a fixture:
Uncaught AssertionError: expected undefined to equal 'Fixtures'
at Interceptor.body (tests/index.js:130:64)
at end (node_modules/nock/lib/request_overrider.js:321:34)
at OverriddenClientRequest.RequestOverrider.req.end (node_modules/nock/lib/request_overrider.js:160:7)
at Request.end (node_modules/request/request.js:1438:14)
at end (node_modules/request/request.js:553:16)
at Immediate.<anonymous> (node_modules/request/request.js:580:7)
2) Thorn Fixtures creation should create a fixture using options.module:
Uncaught AssertionError: expected undefined to equal 'Fixtures'
at Interceptor.body (tests/index.js:167:64)
at end (node_modules/nock/lib/request_overrider.js:321:34)
at OverriddenClientRequest.RequestOverrider.req.end (node_modules/nock/lib/request_overrider.js:160:7)
at Request.end (node_modules/request/request.js:1438:14)
at end (node_modules/request/request.js:553:16)
at Immediate.<anonymous> (node_modules/request/request.js:580:7)
3) Thorn Fixtures creation should create multiple fixtures:
Uncaught AssertionError: expected undefined to equal 'Fixtures'
at Interceptor.body (tests/index.js:259:64)
at end (node_modules/nock/lib/request_overrider.js:321:34)
at OverriddenClientRequest.RequestOverrider.req.end (node_modules/nock/lib/request_overrider.js:160:7)
at Request.end (node_modules/request/request.js:1438:14)
at end (node_modules/request/request.js:553:16)
at Immediate.<anonymous> (node_modules/request/request.js:580:7)
4) Thorn Fixtures linking should create fixtures and link them:
Uncaught AssertionError: expected undefined to equal 'Fixtures'
at Interceptor.body (tests/index.js:452:64)
at end (node_modules/nock/lib/request_overrider.js:321:34)
at OverriddenClientRequest.RequestOverrider.req.end (node_modules/nock/lib/request_overrider.js:160:7)
at Request.end (node_modules/request/request.js:1438:14)
at end (node_modules/request/request.js:553:16)
at Immediate.<anonymous> (node_modules/request/request.js:580:7)
5) Thorn Fixtures linking should retry fixture creation and linking on 401's:
Uncaught AssertionError: expected undefined to equal 'Fixtures'
at Interceptor.body (tests/index.js:495:64)
at end (node_modules/nock/lib/request_overrider.js:321:34)
at OverriddenClientRequest.RequestOverrider.req.end (node_modules/nock/lib/request_overrider.js:160:7)
at Request.end (node_modules/request/request.js:1438:14)
at end (node_modules/request/request.js:553:16)
at Immediate.<anonymous> (node_modules/request/request.js:580:7)
6) Thorn Fixtures linking with pre-existing records should link records:
Uncaught AssertionError: expected undefined to equal 'Fixtures'
at Interceptor.body (tests/index.js:536:68)
at end (node_modules/nock/lib/request_overrider.js:321:34)
at OverriddenClientRequest.RequestOverrider.req.end (node_modules/nock/lib/request_overrider.js:160:7)
at Request.end (node_modules/request/request.js:1438:14)
at end (node_modules/request/request.js:553:16)
at Immediate.<anonymous> (node_modules/request/request.js:580:7)
7) Thorn Agent request methods should send GET request:
Uncaught AssertionError: expected undefined to equal 'Agent-foo'
at Interceptor.body (tests/index.js:882:64)
at end (node_modules/nock/lib/request_overrider.js:321:34)
at OverriddenClientRequest.RequestOverrider.req.end (node_modules/nock/lib/request_overrider.js:160:7)
at Request.end (node_modules/request/request.js:1438:14)
at end (node_modules/request/request.js:566:14)
at Immediate.<anonymous> (node_modules/request/request.js:580:7)
8) Thorn Agent request methods should send POST request:
Uncaught AssertionError: expected undefined to equal 'Agent-foo'
at Interceptor.body (tests/index.js:900:64)
at end (node_modules/nock/lib/request_overrider.js:321:34)
at OverriddenClientRequest.RequestOverrider.req.end (node_modules/nock/lib/request_overrider.js:160:7)
at Request.end (node_modules/request/request.js:1438:14)
at end (node_modules/request/request.js:553:16)
at Immediate.<anonymous> (node_modules/request/request.js:580:7)
9) Thorn Agent request methods should send PUT request:
Uncaught AssertionError: expected undefined to equal 'Agent-foo'
at Interceptor.body (tests/index.js:920:64)
at end (node_modules/nock/lib/request_overrider.js:321:34)
at OverriddenClientRequest.RequestOverrider.req.end (node_modules/nock/lib/request_overrider.js:160:7)
at Request.end (node_modules/request/request.js:1438:14)
at end (node_modules/request/request.js:553:16)
at Immediate.<anonymous> (node_modules/request/request.js:580:7)
10) Thorn Agent request methods should send DELETE request:
Uncaught AssertionError: expected undefined to equal 'Agent-foo'
at Interceptor.body (tests/index.js:940:64)
at end (node_modules/nock/lib/request_overrider.js:321:34)
at OverriddenClientRequest.RequestOverrider.req.end (node_modules/nock/lib/request_overrider.js:160:7)
at Request.end (node_modules/request/request.js:1438:14)
at end (node_modules/request/request.js:553:16)
at Immediate.<anonymous> (node_modules/request/request.js:580:7)
11) Utils login should post to the login endpoint with the right params:
Uncaught AssertionError: expected undefined to equal 'TestHeader'
at Interceptor.body (tests/utils.js:61:60)
at end (node_modules/nock/lib/request_overrider.js:321:34)
at OverriddenClientRequest.RequestOverrider.req.end (node_modules/nock/lib/request_overrider.js:160:7)
at Request.end (node_modules/request/request.js:1438:14)
at end (node_modules/request/request.js:553:16)
at Immediate.<anonymous> (node_modules/request/request.js:580:7)
12) Utils refresh should post to the refresh endpoint with the right params:
Uncaught AssertionError: expected undefined to equal 'TestHeader'
at Interceptor.body (tests/utils.js:102:60)
at end (node_modules/nock/lib/request_overrider.js:321:34)
at OverriddenClientRequest.RequestOverrider.req.end (node_modules/nock/lib/request_overrider.js:160:7)
at Request.end (node_modules/request/request.js:1438:14)
at end (node_modules/request/request.js:553:16)
at Immediate.<anonymous> (node_modules/request/request.js:580:7)
Because it's sometimes far more helpful than just normal THORN_VERBOSE.
Alternatively, we could make it so that THORN_VERBOSE=3 instead triggers NODE_DEBUG=request, so that people don't have to remember more than one environment variable.
Mocha 4.0.0 was released earlier today. None of the changes seem particularly relevant to Thorn, but we should test it and see what will happen.
Date.now
, which is used in #122 for enforcing unique user names, only gives millisecond precision. I've seen tests which gave the same string to two different users because it happened that quickly. (I have yet to see this happen for two different users with the same username, mind you).
So IMO this is cause for us to move to a proper UUID suffix.
In addition to #139, we also need to output the value of response.error
when we receive an invalid response.
response.error.code
is particularly interesting, as it sometimes shows up in cases like this.
See https://dareid.github.io/chakram/jsdoc/global.html#ChakramResponse so the API I'm talking about is clear.
The index.js test file is over 700 lines of code. It is probably a good idea to try to split it up into smaller logical files.
Would mean we don't have to manually run npm outdated
every day.
Just a thought!
Instead of just saying Fixtures
in the THORN_VERBOSE output, we could say one of Fixtures.create
, Fixtures.link
, or Fixtures.cleanup
.
So an example might look something like:
Request 1 Fixtures.create: POST /path/to/sugarcrm/rest/v10/oauth2/token
This might require, or be made simpler by, changing the value of X-THORN headers for these requests.
There's not really any point in doing this for UserAgent
requests, since the function in use there is clear by the HTTP method (GET/POST/etc.)
co-mocha
, and how we're using it is, IMO, probably the most confusing aspect of Thorn usage today.
mocha
invocation).It uses generators in a really weird way, that is probably against the grain of promises (something that should be natively available in Node 8.x), so I feel like we may wish to eventually get rid of it altogether, but for now at least we should at least document it.
@jcsmorais, thoughts on this?
Ever since #140 got merged, these sorts of unique names have been unnecessary and they don't reflect our current testing practice. So I think we can go through and change them.
The Thorn docs example for Fixtures.link is incorrect.
let response = yield Fixtures.link(cachedRecords.Accounts[0], 'accounts_contacts', cachedRecords.Contacts[0]);
The link name isn't accounts_contacts
, it should be contacts
.
accounts_contacts
is the relationship name, not the name of the link.
When one tries to run a considerable amount of tests (was able to reproduce this with 23 different test cases, but I guess it depends on what your machines / instance speed is) eventually admin's session will expire leading to a 401:
{"error":null,"response":{"statusCode":401,"body":{"error":"invalid_grant","error_message":"The access token provided is invalid."},"headers":{"date":"Fri, 17 Feb 2017 20:21:03 GMT","server":"Apache/2.4.18 (Unix) PHP/5.6.27","x-powered-by":"PHP/5.6.27","expires":"Thu, 19 Nov 1981 08:52:00 GMT","cache-control":"no-store","pragma":"no-cache","content-length":"81","status":"401 Unauthorized Unknown","connection":"close","content-type":"application/json"},"request":{"uri":{"protocol":"http:","slashes":true,"auth":null,"host":"localhost","port":80,"hostname":"localhost","hash":null,"search":null,"query":null,"pathname":"/sugar-builds/79/ent/sugarcrm/rest/v10/bulk","path":"/sugar-builds/79/ent/sugarcrm/rest/v10/bulk","href":"http://localhost/sugar-builds/79/ent/sugarcrm/rest/v10/bulk"},"method":"POST","headers":{"Content-Type":"application/json","X-Thorn":"Fixtures","OAuth-Token":"ed35109d-cd45-410b-81ea-dfcfd1a7bb39","accept":"application/json","content-length":477}}},"body":{"error":"invalid_grant","error_message":"The access token provided is invalid."},"jar":{"_jar":{"version":"[email protected]","storeType":"MemoryCookieStore","rejectPublicSuffixes":true,"cookies":[]}},"url":"http://localhost/sugar-builds/79/ent/sugarcrm/rest/v10/bulk","responseTime":809.434959}
The problem seems to be related with the fact that we're not doing any 401 handling for admin users, see here thus it's falling into the missing "else" statement.
I saw this happening at the Fixtures.create, which makes use of _adminLogin
but also checks the oauth-token to see if it's defined already or not here. Even though we're wrapping the requests regarding record creation, we're not doing it for admin login.
1) Metadata Handler generateFieldValue decimals should return a number with the proper number of digits:
TypeError: Cannot read property 'length' of undefined
at Context.it (tests/metadata-handler.js:129:35)
at Test.Runnable.run (node_modules/co-mocha/lib/co-mocha.js:43:16)
% npm install
/Users/rhogg/src/thorn/tests/metadata-fetcher.js
20:35 error Unable to resolve path to module '../dist/metadata-fetcher.js' import/no-unresolved
The reason is that the dist files aren't built yet, so we are referencing something that doesn't exist.
The issue is likely that we need to throw in https://github.com/sugarcrm/thorn/pull/113/files#diff-168726dbe96b3ce427e7fedce31bb0bcR607 if the response is 400 or 500 class.
Start using templates for variable usage within strings, see here.
It's in the JSDoc, but not Docs.md, which is probably where most people would notice it.
Might be a cool way to help us move to ES 6 modules.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.