vasar007 / projectv Goto Github PK

Describe the bug
If we take one Data class and add to it more than one appraiser they will consume objects from single queue and we aren't be able to get all necessary results.

Expected behavior
Need to split queue to several queues which would be unique for appraiser.

Additional context
Please, make sure that issue will be resolved in proper way.

Is your feature request related to a problem? Please describe.
Service become more and more complicated and need to describe all details in deploy guide to avoid redundant questions.

Describe the solution you'd like
Create page with installing instructions in Wiki and add link in README to guide.

Describe alternatives you've considered
None.

Additional context
It is important to keep the guide up to date.

Is your feature request related to a problem? Please describe.
After adding new views of the main Manager, it remains to add utility classes to simplify working with them.

Describe the solution you'd like
Need to create helper classes to create Shell instances of Async and Rx options of ThingAppraiser Service.

Describe alternatives you've considered
None.

Additional context
Builders must be compatible with current BuildingDirector feature. You should implement AppConfig and XDocument builders at least.

Is your feature request related to a problem? Please describe.
Need to add greeting message for new bot users. This can give additional information on how to use the bot and what it is for.

Describe the solution you'd like
We should add new command to process for bot. Also need to remove technical debt and separate cache logic into special class.

Describe alternatives you've considered
None.

Additional context
Don't forget to register new class(-es) in IoC-container in ASP.NET Core.

Is your feature request related to a problem? Please describe.
We should provide default values that could be used instead of environment variables if user does not set it. It can improve developer experience and simplify debugging and configure of the project.

Describe the solution you'd like
Add class providing environment variables OR default values which developers can be set on their own.

Describe alternatives you've considered
Config file would be great addition.

Additional context
Please, also describe this feature in set up guide.

Is your feature request related to a problem? Please describe.
Instead of printing message in console we need to some other outputs (e.g. saving to files, uploading to Google Docs/Tables).

Describe the solution you'd like
Add classes to work with processing data. They should output results for users.

Describe alternatives you've considered
None.

Additional context
Need to upgrade this option in the future.

Is your feature request related to a problem? Please describe.
Need to unite all library managment in one place and control any cases which could happen. It should help to free users from writting superfluous code and reduce library misuse cases.

Describe the solution you'd like
Implement Shell class which should control all internal library classes calls.

Describe alternatives you've considered
None.

Additional context
Replace code in main method with shell's calls.

Is your feature request related to a problem? Please describe.
We have work with local files. However, it's not convenient always work with on local host because there are a lot of data which people could store in the clouds. That's why need to support such input type too.

Describe the solution you'd like
You should read Google Disk API and add classes which can work with it.

Describe alternatives you've considered
At least, Google Disk API support is required.

Additional context
Also it would be great if you add support for another Cloud Platforms :)

Is your feature request related to a problem? Please describe.
Should avoid directly calls of displaying messages to any output methods. We can abstract writing messages to users.

Describe the solution you'd like
Add interface and messages manager class. The first one should specify needed method to work with output messages and the second one should control given message handler class.

Describe alternatives you've considered
None.

Additional context
Need to follow existing manager class interfaces (i.e. avoid strange behaviour in messages manager).

Is your feature request related to a problem? Please describe.
Current pipeline logic strongly connected with other services components implementations (i.e. crawlers and appraisers). That is actually design smell. It is really hard to read and improve or extend pipeline logic.

Describe the solution you'd like
Need to move all TPL Dataflow Pipeline logic to the new library project. And all managers should use this project to configure their part of pipeline (using pipeline builder).

Describe alternatives you've considered
May be we can ressurect ReactX part of service if we can separate pipeline logic.

Additional context
It would be great if you can write tests for the new pipeline library.

Is your feature request related to a problem? Please describe.
It is necessary to provide the service user with the ability to flexibly configure it through the UI and Console.

Describe the solution you'd like
Need to create common settings classes and add helpers for Console and Desktop apps.

Describe alternatives you've considered
None.

Additional context
It is important to consider the possibility of extending the settings.

Is your feature request related to a problem? Please describe.
We use a lot of third-party libs. It can be very dangerous and fragile to continue work with such environment.

Describe the solution you'd like
Need to move all external dependencies to separate adapter assemblies (use Adapter design pattern).

Describe alternatives you've considered
None.

Additional context
Do not forget to ensure that services will still work after changes.

Is your feature request related to a problem? Please describe.
We should add contracts verification according to existing docs. If there are no docs in wiki about some conditions and remarks, we must create wiki pages with detailed explanation.

Describe the solution you'd like
Clarify all methods contracts and implement verifications for them. We can use our extensions helpers to avoid code duplication or use some third-party library to work with contracts.

Describe alternatives you've considered
We also can write XML docs in the code and generate HTML version of the docs by special tools.

Additional context
Add any other context or screenshots about the feature request here.

Is your feature request related to a problem? Please describe.
Need to add more option to work with movie data.

Describe the solution you'd like
Implement crawler and appraiser to interact with OMDB data and integrate them into our service.

Describe alternatives you've considered
None.

Additional context
You may use some third-party lib to work with OMDB API.

Is your feature request related to a problem? Please describe.
A lot of people are using toplist to sort different things. We can add feature that can simplify work with this toplists.

Describe the solution you'd like
Need to add new view, view-model and model which can interact with toplists and their structure. It should be seem like plain notepad with pinned blocks.

Describe alternatives you've considered
None.

Additional context
It would be nice if we can implement several kinds of toplists.

Is your feature request related to a problem? Please describe.
We use NLog to work with logging stuff. But there are several inconveniences that complicate logging things.

Describe the solution you'd like
Extend format of logging to file and remove non-informative messages. Also improve current adapter for NLog lib because now this adapter is implemented in a wrong way.

Describe alternatives you've considered
None.

Additional context
Check that changes work better than current state.

Is your feature request related to a problem? Please describe.
Now everyone has an access to REST API. So, we should add a new option to call REST API only for authenticated users.

Describe the solution you'd like
Need to use ASP.NET tools to authenticate users.

Describe alternatives you've considered
None.

Additional context
None.

Describe the bug
ThingAppraiser (TA) data flow pipeline consist of Inputters, Crawlers, Appraisers and Outputters. If one of this components throws any exceptions or fall service are not able to process this case and deadlock happens.

To Reproduce
Steps to reproduce the behavior:

1. Add throw new Exception(); at the first line of implementation of any crawler (or any other component). Yes, it sounds wierdly but image that some of your services have fall;
2. Start client and TA services;
3. Execute things appraising for service which component you has been "broken" just now;
4. See infinite request processing.

Expected behavior
Need to add exceptions handling and process them in proper library layout. Also all exceptions must be logged with their stacktraces!

Additional context
Add exception processing for all components of the ThingAppraiser service and try to give clear and descriptive error messages.

Is your feature request related to a problem? Please describe.
CI & CD tools are great addition to almost all projects. They have a lot of advantages and we should install them to our repository.

Describe the solution you'd like
Travis CI is a one of the best choices for this type of work. For windows part you can consider AppVeyor.

Describe alternatives you've considered
None.

Additional context
At least, CI should support Windows but it would be great if we can build on all actual platforms (Windows, Linux, macOS).

Is your feature request related to a problem? Please describe.
Now we have calculation logic inside appraisers classes which is not the best option at this time. It is difficult to change calculation formula and/or inject the new one. In addition, we cannot write proper tests for such infrastructure.

Describe the solution you'd like
Need to extract rating calculation logic into new hierarchy of various strategies. It should be have common interface that let us to change implementation as we want.

Describe alternatives you've considered
None.

Additional context
Do not forget to refactor all appraisers and inject proper strategies to all of them.

Is your feature request related to a problem? Please describe.
There are lots of hard-coded parameters in classes which isn't a great decision. Need to fix this code smell and remove magic constants and strings (as much as possible).

Describe the solution you'd like
Find and change logic of constants initialization. Simply, you can delegate this to Shell or Builder class provide appropriate constructor or set methods for variables (constructor way is more preferable).

Describe alternatives you've considered
I think that optional parameters would be alternative but don't overuse it.

Additional context
These changes can impact the performance of the service. That's why we need to make performance tests after removing magic constants.

Is your feature request related to a problem? Please describe.
There are awesome lib to map DTO (from REST or DB) with internal models which called AutoMapper. We could be use it instead of writing our own mappers.

Describe the solution you'd like
Try to install AutoMapper and use it for the small number of DTO to convert them into models. If this would be work nice you can try to replace all mappers with lib.

Describe alternatives you've considered
May be there are other mapper libraries which I am not familiar with.

Additional context
Please, make sure that all changes do not break or corrupt our data in pipelines (or in communication between services). Besides, create benchmarks and measure performance with and without mapper lib.

Is your feature request related to a problem? Please describe.
In order to save traffic and computing resources, we can implement a handler that will check the availability of data in the database and report what information is already there, and what we need to get from Internet services (i.e. TMDB).

Describe the solution you'd like
Need to create class which can monitor data base and provide list of available data to reuse. Also if we want to force clean and fill DB again, then there should be option to do such action.

Describe alternatives you've considered
None.

Additional context
Not forget to test solution with different cases.

Is your feature request related to a problem? Please describe.
Suppose, you have a content of any type (i.e. movies, videos, images, e.t.c.) on your hard drive or cloud. At one point, this content can accumulate a lot and it would be impossible to find necessary movie/video/image in that content hell in a conveniently short time. At at this time ContentFinder feature of ThingAppraiser project can help you a lot!

Describe the solution you'd like
Need to develop library that can find specified type of content inside folder(-s). Result type should be dictionary in form:

{
	"content_name": [ "path1", "path2", ... ],
	...
}

Then other part of service can process this results and produce user-friendly model to work with.

Describe alternatives you've considered
It would be intersting to find similar solutions.

Additional context
Please, provide tests for your implementation too.

Is your feature request related to a problem? Please describe.
We have a lot of classes now. If we want to keep project in stable state we should take care of adding tests.

Describe the solution you'd like
You should explore and find good C# Test Framework, describe it pros and cons and add couple of tests for beginning.

Describe alternatives you've considered
None.

Additional context
Also you can integrate test process with potential CI tool.

Is your feature request related to a problem? Please describe.
Need to support undocummented API for Steam. It would be very useful have a opportunity to work with this service API.

Describe the solution you'd like
Implement crawlers and appraisers with other stuff to work with Steam API. As an example you can use TMDb components.

Describe alternatives you've considered
None.

Additional context
It would be great if we can find any good docs about Steam API.

Is your feature request related to a problem? Please describe.
Need to change configuration without recompiling all assembly.

Describe the solution you'd like
I think that we should implement ConfigParser which would parse config file and set internal flags for Shell to create specified classes and setup some other stuff.

Describe alternatives you've considered
Don't suppose another option.

Additional context
None.

Is your feature request related to a problem? Please describe.
Library classes have a lot of public API. It is possible that we will have a mess in the near future.

Describe the solution you'd like
Need to reconsider accessibility levels of methods and classes and convert them to new level which should be as strict as possible. Also contract checkers must be added to ALL public methods. That is let us to catch errors much earlier.

Describe alternatives you've considered
May be some classes could be removed completely (be careful with permanent classes removal from solution).

Additional context
It would be nice if all public API had documentation pages in our Wiki.

Is your feature request related to a problem? Please describe.
This class should display notifications and error reports directly to console. It should be follow message handler interface.

Describe the solution you'd like
Need to abstract and unite all message writting in this class. Simply implement interface of message handler.

Describe alternatives you've considered
None.

Additional context
Related with #13 issue.

Is your feature request related to a problem? Please describe.
Need to store temporary and processed data during crawling process.

Describe the solution you'd like
Add database and classes to interact with data layer.

Describe alternatives you've considered
None.

Additional context
We should add MS-SQL firstly because of internship task but can change this one later.

Is your feature request related to a problem? Please describe.
We have now preview version of .NET Core 3.0. There are a lot of new features and different imrpovements.

Describe the solution you'd like
Try to install preview version of .NET Core 3.0 and update all projects in this solution. Do not forget to replace preview version with stable later.

Describe alternatives you've considered
None.

Additional context
Aslo need to remove .NET Framework 4.7.2 (use .NET Core WPF instead of .NET Framework WPF) and update .NET Standard version of all libraries from 2.0 to 2.1.

Is your feature request related to a problem? Please describe.
A lot of public methods operate in terms of concrete data structure impelemntations. It could be lead to focus on specific implementation in our logic. In addition, such code is harder to improve and refactor.

Describe the solution you'd like
Replace all concrete data structure implementations with interfaces. I.e. List<T> you could replace with IReadOnlyList<T>, Dictionary<T> -> IReadOnlyDictionary<T> and so on.

Describe alternatives you've considered
None.

Additional context
Be sure that this changes do not break serialization of DTO or other code contracts.

Is your feature request related to a problem? Please describe.
We should research potential issue related to performance and memory consumption.

Describe the solution you'd like
Use profiling tools to find critical places in ProjectV serivces. All found problems should be documented as issues in this repository.

Describe alternatives you've considered
None.

Additional context
Please, provide detailed report and write stress tests.

Is your feature request related to a problem? Please describe.
We need to add support of input files and some other options.

Describe the solution you'd like
Implement work with files and services (e.g. Google Docs/Tables).

Describe alternatives you've considered
None.

Additional context
Files support is required, others are optional.

Is your feature request related to a problem? Please describe.
Need to add documentation because there are a lot of classes in service now and their number will be grown.

Describe the solution you'd like
Add documentation for all service classes according to Microsoft XML documentation guide.

Describe alternatives you've considered
None.

Additional context
Please check typos and correctness of your docs strings.

Is your feature request related to a problem? Please describe.
Now we can get required data from API with movie data bases (MDB). Presently, we need to process this data and sort input films based on ratings.

Describe the solution you'd like
Add some classes which implement data processing. Crawler classes can get data from MDB and parse them to list of data-classes. All data-classes inherit from base class Movie. It contains all necessary data to analyze.

Describe alternatives you've considered
All alternatives need to discuss in the comment section.

Additional context
None.

Is your feature request related to a problem? Please describe.
Need to implement multi-threading data processing. Such option allows us to scale service and remove bottlenecks with long responses from services with data storage (TMDB, IMDB, etc).

Describe the solution you'd like
We consider several options. First we can implement TPL DataFlow with async/await approach.

Describe alternatives you've considered
Second option includes Reactive Extensions. I think it would be more complex than implement async/await solution.

Additional context
Also need to add helper classes to easy and out-of-box work with multi-threading.

Is your feature request related to a problem? Please describe.
Need to implement saving program reports and logs into some files in the working directory for examle. It should help to debug app and decrease spending time to fix errors.

Describe the solution you'd like
Create class which can save logging messages and error reports to files and make a call to MessageHandler class to display problem.

Describe alternatives you've considered
You can (and must) use some third-party library if it implements needed behaviour.

Additional context
Writting messages to console should be replaced by calling logger instance in existing classes.