meanpug / folding-together Goto Github PK
View Code? Open in Web Editor NEWDemocratizing folding@home (and potentially other networks like rosetta@home)
License: MIT License
Democratizing folding@home (and potentially other networks like rosetta@home)
License: MIT License
Function to start the CloudFormation stack with any required input parameters
This function will be responsible for tracking the available balance for running compute over time. As time passes this value should go down and help represent a near-real-time balance of the funds available for a particular donor.
For example, if we run the function every minute and the target instance currently costs $2.40/hr, we should decrement by $0.04
We can report on team stats like:
Relevant endpoints:
Sample Response:
{
"founder": "CygnusXI",
"id": 224497,
"logo": "http://i.imgur.com/PIdLHV1.jpg",
"name": "Curecoin",
"rank": 1,
"score": 1114003226872,
"url": "http://curecoin.net",
"wus": 25490973
}
If this gets really popular, we will have to worry about service quotas (soft limits) such as:
Database to track balance on a per-donor basis
Thinking DynamoDB with strong consistency
Currently we default to no team. A donor should be able to specify a team.
The donor-team combo can be a unique value in the balance tracker.
We want a "failsafe" mechanism to prevent o proactively alert when costs are accruing without the donations to back them. For example, if an instance fails to terminate when scheduled, we want to find out before it runs for several hours and generates costs that the project maintainers will have to bear.
As a way to improve #4, we should publish to the SQS queue async (through either celery worker or migration to asgi+asyncio)
Map CloudFormation stack ids to donors in the ddb table
Function to terminate the stack
On the detail page, we can include information like:
Trigger: probably cron #26. Frequency as often as one minute but less frequent will also work as long as the drawdown is proportional to the execution frequency.
Steps:
This will be a requirement for #2 .
We have two options here, and I'm not yet sure which is the right one:
For this, we back compute with EC2 resources and an ASG, so we'll need:
We back compute with Fargate resources. A pre-requisite here is that we can get folding@home working properly in a container and the container can be juiced with enough access to the host compute resources:
The API should accept a donor ID and retrieve the status (queued vs. processing vs. finished
) for the donor
Pick a frequency and write a cron expression for it in a CloudWatch event that can be used for the state machine trigger
The gateway simply needs to allow for basic user registration and payments. Upon successful payment, it can queue a task to schedule compute for the available credits.
cc @atkinsonm
Helpful Links:
Goal
To get the folding@home software set up and working on a single EC2 tiny instance. Documentation of any issues / steps can be attached as comments
I would like to have a way to measure whether an instance is actually running folding@home jobs. If an instance (or container) is running but sitting idle (because it hasn't received assignment, potentially do to upstream saturation), it's not useful to anyone and should be hibernated or terminated to save resources.
This is the function which runs when a donation is received. It is responsible for identifying the donor username and amount and updating the available balance for that donor.
There is a hard limit of 200 CloudFormation stacks per region. If this solution gets popular, the 1 stack per donor will no longer be a viable model.
We can avoid this with custom calls against the EC2 API. We might be able to reuse bits like security groups, instance profiles, AMIs, etc.
Relates to #17
currently, when someone makes an anonymous donation through the Anonymous user, the balance of the associated anonymous record is incremented appropriately, however because we don't update the stack ID to empty
upon completion and subsequent termination, a new stack is never launched.
The proposed fix is to update stack ID to empty
upon termination of FAH instances, which we should probably be doing anyway
Essentially, this service should understand how many blocks of time have been purchased and - based on this - adjust the ASG parameters. For instance, assume donor A purchased 20m of time and donor B 40m of time. The scheduler would then understand that an instance may be added to the ASG for 60m before a new scaling event. Assume then that a new donor purchases, 60m of time. Rather than line up blocks in serial, a new instance should be scheduled immediately and the ASG scaled up to 2 for 1 hour (rather than 1 for 2 hours).
The TlDr here is, schedule in 60m increments
I think it would be cool to have a FoldingTogether team to serve as a way to showcase the contributions from this team and its donors and other contributors.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.