What's the Testing Story for apps using Azure SDKs? about sdk HOT 3 CLOSED

@JonathanGiles - please add guidelines on recommendations on how to implement mockable clients. The current guidance is a copy of the general guidelines and not sufficient.

from sdk.

Notes from Architecture Board:

Recording: https://msit.microsoftstream.com/video/dab58b12-8ebe-49ed-8ace-876af461db84

PR #894 (Draft)

1. 3 discussion points

• Jeffrey notes about recordings and how they work. There are many things we have to ignore - it generally works, but someone has to identify whether a field is valid or needs to be ignored.
• How to deal with situations where the service team breaks the endpoint APIs after tests are recorded. Isn't that a breaking change, or point to a breaking change? The guideline should almost be the reverse (after GA) since our intent is to avoid breaking changes - there are situations where the recordings need to change, but it shouldn't be because of a breaking change.

Additional comments:

• When should we be running live tests? We should add a guideline for this - at least before every GA release, and use different service API versions.
• Maybe we should establish guidelines on when to mock, with the default to not mock. e.g. fault injection appears to be one area where we heavily rely on mocks
• Who needs to decide what "appropriate" code coverage is actually reasonable. There should be a test for every single publically exposed operation. Also, error handling needs to be tested. Focus on clients rather than models. Payloads that can be serialized and deserialized should be tested.
• Reaching 100% code coverage is next to impossible - how do you select what to add to improve coverage.
• One guidelines that might be added regarding recordings is that perhaps you should exclude sensitive information from the recording.

from sdk.

Completed and merged into general guidelines.

from sdk.