Code Monkey home page Code Monkey logo

tickgrinder's Introduction

TickGrinder Algorithmic Trading Platform

Join the chat at https://gitter.im/TickGrinder/Lobby

TickGrinder is a high performance algorithmic trading platform written primarily in Rust. It is designed with the goal of efficiently processing event-based market data as quickly as possible in order to automatically place and manage trades.

Currently this platform is only compiles and runs on Linux-based systems. Windows functionality is planned for the future but no set schedule has been defined for its implementation.

Overview

The basis of the platform is written in Rust. It consists of several distinct modules that operate independently but communicate with each other via a custom messaging protocol implemented on top of Redis Pub/Sub. It is designed to be extensible and robust, capable of being used to trade any market consisting of event-based streaming Tick data.

Disclaimer: This platform is currently in active, early pre-alpha development and is in no way ready for any kind of trading. I do not take any responsibility for your trading actions using this platform nor any financial losses caused by errors in this application.

Tick Processors

The primary module is the Tick Processor. Multiple tick processors can be spawned, one for each symbol/data stream that is being processed by the platform. Their purpose is to convert live data into trading signals as quickly as possible.

Each time a tick arrives, a series of conditions are evaluated. These conditions can be anything: a SMA crossing a threshold, the current timestamp being greater than a certain number, any evaluable expression that you can program can serve as a condition. These conditions are designed to be dynamically set by the Optimizer module during trading operations.

Optimizer/Strategies

The Optimizer is a module that controls the conditions evaluated by the Tick Processors to open trades and interact with the broker API. Only one Optimizer is meant to run at once and it interacts with all Tick Processors that may be alive in the platform.

Optimizers, using whatever strategies you define, set the parameters and enable/disable trading conditions on the Tick Processors dynamically. They accomplish this by sending and receiving Commands to and from the Tick Processors, interacting with the database, or using any external data sources you may find useful. This application currently uses PostgreSQL as the main storage backend.

The strategies that are evaluated by the optimizer are meant to be written by you, the user! I'm keeping my strategies secret, but I will provide a sample strategy in the future for reference.

MM (Management/Monitoring) Interface

The MM is the interface between the platform and the user. It contains controls for manually managing platform components, monitoring bot activity, starting backtests, and pretty much everything else. It exists in the format of a simple NodeJS Express webserver and talks with the main communication system of the platform using a Websocket<->Redis bridge system.

It includes a custom charting module using Highcharts that can be extended with custom indicators to produce truly specialized charts for bot data. The configurations for these charts can be compiled down into JSON-encoded Macro strings which can then be loaded to instantly re-create charts with unique data sources and settings.

Broker APIs + Data Downloaders

Several scripts/utilities are included to interact with the FXCM Foreign Exchange API to create a live data feed and access historical market data. Using scripts located in the tick_writer and data_downloaders directories, this application can be controlled through Redis and used as a primary data source for the trading platform.

The data downloader script saves tick data for a currency pair into CSV files, downloading chunks of data automatically. The tick writer script does the same thing but processes live ticks sent over Redis instead. As I mentioned before, more detailed documentation can be found in the respective directories.

Installation

Requirements:

  • Rust/Cargo Nightly
  • NodeJS v5.0.0 or greater
  • Redis
  • PostgreSQL
  • libboost headers (sudo apt-get install libboost-all-dev)

After cloning the repository, you'll need to copy all instances of files named conf.default.rs, conf.sample.js, or anything similar to conf.js/rs in the same directory and fill out their values as appropriate.

After ensuring that you have a Redis and PostgreSQL server running and accessible to the program, you can make sure that everything is set up correctly by running make test in the root directory of the project. This will automatically pull down all needed dependencies, attempt to build all modules, and run all included tests. Any encountered errors will be printed to the console and you can use them to debug any issues you're having with the setup.

To use the live trading and data downloading functionalities, you'll need to set up the FXCM Java Application (located in the TickRecorder directory). Detailed setup and installation instructions for that are located in that folder. Documentation for the data downloader and tick_writer scripts can also be found in their respective directories.

Contributing

This is an open source project project, so you're more than welcome to fork it and customize it for your own needs.

As for contributing to master, I'm very happy to merge pull requests containing syntax improvements, bugfixes, or small stuff like that. However, for more significant things such as rewriting large segments of code or adding new features, please file an Issue or contact me privately before putting a lot of work in.

In addition, I will not accept any pull requests solely consisting of stylistic changes such as running files through rustfmt or linting the platform with Clippy. I'm aware that some of the choices in syntax and style I've used aren't 100% "rusty," but I made the choices I made for a reason. Things like fixing typos, adding in more clear or detailed comments/documentation, or re-implementing functions in a more concise or efficient method are very welcome, however.

Closing Remarks

If you've got any feedback or comments on the project, I'd love to hear it! I'm always working on developing my skills as a programmer, so any sage advice from seasoned veterans (or questions from eager beginners) are very welcome.

If you find this project useful, exciting, or have plans to use this in production, please let me know! I'd maybe be willing to work with you to make sure that your needs are met and improve the platform in the process.

tickgrinder's People

Contributors

ameobea avatar gitter-badger avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tickgrinder's Issues

Get the build process working on OSX

Since I do most of my development on mac these days, it will be very helpful to be able to build and run the platform on it. In addition, many people use mac and being able to have it install there without having to go through extra hassle is a great bonus.

In the future, I'd like to get it building on Windows as well, but as I don't really have a way to test that (and it's not a priority for me right now) that will have to wait for another time.

Update Dependencies

All dependencies in use by the platform should be updated to their latest versions. The goal is to peg the entire platform to one version of Rust nightly which can then be incrementally updated as development continues.

don't have any conf.rs/ conf.default.rs file

Compiling tickgrinder-util v0.1.0 (/home/rustmann/RustProjects/tickgrinder/util)
error[E0583]: file not found for module conf
--> src/lib.rs:27:9
|
27 | pub mod conf;
| ^^^^
|
= help: name the file either conf.rs or conf/mod.rs inside the directory "src"

error: aborting due to previous error

For more information about this error, try rustc --explain E0583.
error: Could not compile tickgrinder-util.

To learn more, run the command again with --verbose.
Makefile:171: recipe for target 'test' failed
make: *** [test] Error 101

Does not compile

$ make test
# copy libstd to the dist/lib directory if it's not already there
if [[ ! -f dist/lib/$(find $(rustc --print sysroot)/lib | grep -E "libstd-.*\.so" | head -1) ]]; then \
	cp $(find $(rustc --print sysroot)/lib | grep -E "libstd-.*\.so" | head -1) dist/lib; \
fi;
cd configurator && LD_LIBRARY_PATH="../../dist/lib" RUSTFLAGS="-L ../../util/target/debug/deps -L ../../dist/lib -C prefer-dynamic" cargo test --no-fail-fast
   Compiling bit-set v0.4.0
   Compiling cursive v0.6.4
   Compiling toml v0.4.5
   Compiling rand v0.3.19
   Compiling termion v1.5.1
   Compiling num-integer v0.1.35
   Compiling serde_json v0.8.6
   Compiling num-complex v0.1.41
   Compiling indoc v0.1.15
   Compiling num-iter v0.1.34
   Compiling num-bigint v0.1.41
   Compiling chan v0.1.19
error[E0061]: this function takes 2 parameters but 1 parameter was supplied
  --> /home/caio/.cargo/registry/src/github.com-1ecc6299db9ec823/indoc-0.1.15/src/lib.rs:87:30
   |
87 |             let unindented = parse::str_lit(&unindent(&name.as_str()));
   |                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected 2 parameters

error: aborting due to previous error

error: Could not compile `indoc`.
warning: build failed, waiting for other jobs to finish...
error: build failed
make: *** [Makefile:173: test] Error 101

Probably related to dtolnay/indoc#10

Example or test

Hello~I just visited here. It looks cool project.
Is there some way to test this, or sample project?

Instance Spawning Network Delay

When spawning instances, it's possible that high network delay may cause the instance to take a while to actually spawn, leading the Spawner to think it's dead before it's actually started listening for Ping commands to respond to. The Spawner should put the instance in a Pending queue and either give it a long time to spawn or wait for a Successfully Spawned message from the instance.

Create Dockerfile

Users should be able to set up and run the entire platform through Docker. This will be useful for the continuous integration and will make things like automatic deployments much easier as well.

Set up continuous integration

This project could really benefit from some kind of CI that automatically builds the platform in its entirety and runs the full battery of tests. This will require a significant amount of effort but will probably be worth it in the long run.

The first step should be to get all of the tests to pass successfully. Tests involving functionality that no longer exists or must be changed significantly should be removed or disabled until such a time as they can be re-implemented correctly.

Ideally, an automatic deployment system such as now.sh would be set up as well with a full demo version of the platform, but this isn't necessarily in the scope of this issue.

Implement more advanced client-side logging

Currently, the log viewer in the MM is a very simple implementation, simply listening to messages arriving over the wire and adding them to the log table.

I'd love to implement a react-virtualized approach with an infinitely-scrolling list that's populated from the PostgreSQL server. I honestly can't remember very well just how far along it was in development, but it should also have full support for sorting, filtering, etc.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.