veriorpies / minecloud Goto Github PK
View Code? Open in Web Editor NEWAn AWS CDK project to set up an almost free on-demand multiplayer server (Minecraft, Terraria, and more...) for a Discord community in just a few minutes
License: MIT License
An AWS CDK project to set up an almost free on-demand multiplayer server (Minecraft, Terraria, and more...) for a Discord community in just a few minutes
License: MIT License
After this task (#55), MineCloud should be feature completed for adding new multiplayer game support.
Add more supported multiplayer game server and create SOPs for adding a new game.
Is your feature request related to a problem? Please describe.
Currently, every Discord BOT are having the same command prefix - mc
.
Which will become messy easily when there's more than one game deployed in the Discord Server.
Describe the solution you'd like
In the configuration package for each game, each BOT should have their own prefix.
Describe alternatives you've considered
N/A
Additional context
Add any other context or screenshots about the feature request here.
N/A
Is your feature request related to a problem? Please describe.
Every time the Ec2 machine starts, a script could be run to update a configurable DNS record.
Describe the solution you'd like
I own a domain which I would like to use to make it easier to connect to the server by creating a record like minecraft.mydomain.cloud.
In the guide How to automatically update IP addresses a script is run at each boot to update a DNS record with the new IP address. The DNS name is added as a tag to the EC2 instance.
This could also be implemented as a Linux systemd service
An alternative solution would be to integrate the DNS update functionality into the Lambda function which posts the IP address into the Discord channel.
Whatever solution is implemented, the script could look at the EC2 instance tags and if there is a DNS_RECORD tag present it could execute the update Route53 dns record functionality. This would make this an optional feature, which can be controlled by giving an optional environment variable DNS_RECORD_TAG=minecraft.mydomain.cloud
Write CDK code to setup the Lambda to interact with the EC2 instance.
The lambda code is ready here => https://gist.github.com/314pies/3aa3a559cc1c665c796ea9b0e4c0f50a
(we will want to refactor it later btw...)
Test
Make sure it can properly spin up ec2 instance.
Document
N/A
At this point. We should be able to have a E2E working project.
And we should start working on writing an user friendly Readme.
If the command hit the lambda endpoint and the lambda is still cold, it will result in time-out (because our lambda also need to call aws API to spin-up EC2...etc)
Right now how we solved it is by detecting whether the lambda is cold, and prompt user to try again if it's - probably no big deal at most of the time.
This issue should be solvable by utilizing a Discord API/feature that can offer response first and update it later.
Right now we are using Discord for spin up server, which is cool.
But might be also pretty cool if we can provide an alternative such as a Web interface.
If you want this feature, please react to this issues so we know you need it!
Describe the bug
When there's space it the line, each segment of the line will be considered as a individual path, causing the copying behavior failed
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
Smartphone (please complete the following information):
Additional context
Add any other context about the problem here.
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Refactor discord interaction lambda to address cold start issue and make it less spaghetti.
Lambda code: https://github.com/VeriorPies/Minecloud/tree/main/lambda/discord_interactions_endpoint
Deploy the stack and make sure the it doesn't break anything
Add steps to the Readme
to setup CDK dev environment.
I tried deploying with a brand new AWS account.
I do have multiple AWS account credentials on my laptop so I used the correct profile (passing the --profile) tag
USER@xxx-MacBook-Pro MineCloud-1.1.0 % node --version
v18.4.0
USER@xxx-MacBook-Pro MineCloud-1.1.0 % aws --version
aws-cli/2.7.9 Python/3.9.13 Darwin/20.5.0 source/x86_64 prompt/off
xxx@xxx-MacBook-Pro MineCloud-1.1.0 % npx cdk deploy --profile minecraft-discord
Bundling asset MineCloud/MineCloud_discord_commands_register_lambda/Code/Stage...
cdk.out/bundling-temp-4e685f68940a765a4ed4847d4d773ddf7e2d7dfb3afa6065170b3443da43b77d/index.js 370.3kb
⚡ Done in 22ms
Bundling asset MineCloud/MineCloud_discord_interactions_endpoint_construct/MineCloud_discord_interactions_endpoint_lambda/Code/Stage...
cdk.out/bundling-temp-6626f17e65c370d5c0f9f5f0aac55e2ce76f0fb27a8fa4ff20c2220cf3e6ca2c/index.js 18.7mb ⚠️
⚡ Done in 152ms
Bundling asset MineCloud/MineCloud_discord_interactions_endpoint_construct/MineCloud_discord_command_processor_lambda/Code/Stage...
cdk.out/bundling-temp-a4eb7f2edfaee3b0511d1ca7be7999344e11db9698f37ce9f2759340055524ce/index.js 19.0mb ⚠️
⚡ Done in 199ms
✨ Synthesis time: 7.45s
MineCloud: building assets...
[0%] start: Building 97817db928c43d4646fed3437ff70ca3fe8d1ca5d5538e80726641d669c244fd:<AWS_ACC_ID>-ap-south-1
[0%] start: Building 0e22f72895a5bd555e2ce81d5dc6b7d01ac1835cd925dd80276ba2dad75c13fb:<AWS_ACC_ID>-ap-south-1
[0%] start: Building 903be65b9594a9f17d7120273620367e7830b6d087fc9591f31dc5a080049e46:<AWS_ACC_ID>-ap-south-1
[0%] start: Building 2b36d5d4dcb60993ddf36ecf68feed35ecbc70c3b92dcd77638a2c876866ad88:<AWS_ACC_ID>-ap-south-1
[0%] start: Building 85bb78ae8c7c9ad2d7d8a193f0ee234de6d82fdfba52da88120ea46ad54feea4:<AWS_ACC_ID>-ap-south-1
[0%] start: Building e30e6151fa91fad623f41b3e17ed2885697fde9fd59c95edbad26c7ec837a58a:<AWS_ACC_ID>-ap-south-1
[16%] success: Built 97817db928c43d4646fed3437ff70ca3fe8d1ca5d5538e80726641d669c244fd:<AWS_ACC_ID>-ap-south-1
[33%] success: Built 0e22f72895a5bd555e2ce81d5dc6b7d01ac1835cd925dd80276ba2dad75c13fb:<AWS_ACC_ID>-ap-south-1
[50%] success: Built 903be65b9594a9f17d7120273620367e7830b6d087fc9591f31dc5a080049e46:<AWS_ACC_ID>-ap-south-1
[66%] success: Built 2b36d5d4dcb60993ddf36ecf68feed35ecbc70c3b92dcd77638a2c876866ad88:<AWS_ACC_ID>-ap-south-1
[83%] success: Built 85bb78ae8c7c9ad2d7d8a193f0ee234de6d82fdfba52da88120ea46ad54feea4:<AWS_ACC_ID>-ap-south-1
[100%] success: Built e30e6151fa91fad623f41b3e17ed2885697fde9fd59c95edbad26c7ec837a58a:<AWS_ACC_ID>-ap-south-1
MineCloud: assets built
MineCloud: deploying... [1/1]
[0%] start: Publishing 97817db928c43d4646fed3437ff70ca3fe8d1ca5d5538e80726641d669c244fd:<AWS_ACC_ID>-ap-south-1
[0%] start: Publishing 0e22f72895a5bd555e2ce81d5dc6b7d01ac1835cd925dd80276ba2dad75c13fb:<AWS_ACC_ID>-ap-south-1
[0%] start: Publishing 903be65b9594a9f17d7120273620367e7830b6d087fc9591f31dc5a080049e46:<AWS_ACC_ID>-ap-south-1
[0%] start: Publishing 2b36d5d4dcb60993ddf36ecf68feed35ecbc70c3b92dcd77638a2c876866ad88:<AWS_ACC_ID>-ap-south-1
[0%] start: Publishing 85bb78ae8c7c9ad2d7d8a193f0ee234de6d82fdfba52da88120ea46ad54feea4:<AWS_ACC_ID>-ap-south-1
[0%] start: Publishing e30e6151fa91fad623f41b3e17ed2885697fde9fd59c95edbad26c7ec837a58a:<AWS_ACC_ID>-ap-south-1
[16%] success: Published 85bb78ae8c7c9ad2d7d8a193f0ee234de6d82fdfba52da88120ea46ad54feea4:<AWS_ACC_ID>-ap-south-1
[33%] success: Published 2b36d5d4dcb60993ddf36ecf68feed35ecbc70c3b92dcd77638a2c876866ad88:<AWS_ACC_ID>-ap-south-1
[50%] success: Published 97817db928c43d4646fed3437ff70ca3fe8d1ca5d5538e80726641d669c244fd:<AWS_ACC_ID>-ap-south-1
[66%] success: Published 0e22f72895a5bd555e2ce81d5dc6b7d01ac1835cd925dd80276ba2dad75c13fb:<AWS_ACC_ID>-ap-south-1
[83%] success: Published 903be65b9594a9f17d7120273620367e7830b6d087fc9591f31dc5a080049e46:<AWS_ACC_ID>-ap-south-1
[100%] success: Published e30e6151fa91fad623f41b3e17ed2885697fde9fd59c95edbad26c7ec837a58a:<AWS_ACC_ID>-ap-south-1
MineCloud: creating CloudFormation changeset...
4:08:05 PM | CREATE_FAILED | AWS::CloudFormation::CustomResource | MineClouddiscordcommandsregisterresource
Received response status [FAILED] from custom resource. Message returned: Error: 2023-05-07T10:38:04.485Z c0c2fd89-6da0-4857-a322-43bba04d23b8 Task timed out after 3.01 seconds
Logs: /aws/lambda/MineCloud-MineClouddiscordcommandsregisterlambda95-n8pTPd6Hw3t2
at invokeUserFunction (/var/task/framework.js:2:6)
at processTicksAndRejections (internal/process/task_queues.js:95:5)
at async onEvent (/var/task/framework.js:1:369)
at async Runtime.handler (/var/task/cfn-response.js:1:1573) (RequestId: cad00625-ab3f-4c88-b864-ee8ec4fd5dc5)
❌ MineCloud failed: Error: The stack named MineCloud failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Received response status [FAILED] from custom resource. Message returned: Error: 2023-05-07T10:38:04.485Z c0c2fd89-6da0-4857-a322-43bba04d23b8 Task timed out after 3.01 seconds
Logs: /aws/lambda/MineCloud-MineClouddiscordcommandsregisterlambda95-n8pTPd6Hw3t2
at invokeUserFunction (/var/task/framework.js:2:6)
at processTicksAndRejections (internal/process/task_queues.js:95:5)
at async onEvent (/var/task/framework.js:1:369)
at async Runtime.handler (/var/task/cfn-response.js:1:1573) (RequestId: cad00625-ab3f-4c88-b864-ee8ec4fd5dc5)
at FullCloudFormationDeployment.monitorDeployment (/Users/USER/Downloads/MineCloud-1.1.0/node_modules/aws-cdk/lib/index.js:380:10236)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async deployStack2 (/Users/USER/Downloads/MineCloud-1.1.0/node_modules/aws-cdk/lib/index.js:383:145458)
at async /Users/USER/Downloads/MineCloud-1.1.0/node_modules/aws-cdk/lib/index.js:383:128776
at async run (/Users/USER/Downloads/MineCloud-1.1.0/node_modules/aws-cdk/lib/index.js:383:126782)
❌ Deployment failed: Error: Stack Deployments Failed: Error: The stack named MineCloud failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Received response status [FAILED] from custom resource. Message returned: Error: 2023-05-07T10:38:04.485Z c0c2fd89-6da0-4857-a322-43bba04d23b8 Task timed out after 3.01 seconds
Logs: /aws/lambda/MineCloud-MineClouddiscordcommandsregisterlambda95-n8pTPd6Hw3t2
at invokeUserFunction (/var/task/framework.js:2:6)
at processTicksAndRejections (internal/process/task_queues.js:95:5)
at async onEvent (/var/task/framework.js:1:369)
at async Runtime.handler (/var/task/cfn-response.js:1:1573) (RequestId: cad00625-ab3f-4c88-b864-ee8ec4fd5dc5)
at deployStacks (/Users/USER/Downloads/MineCloud-1.1.0/node_modules/aws-cdk/lib/index.js:383:129083)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async CdkToolkit.deploy (/Users/USER/Downloads/MineCloud-1.1.0/node_modules/aws-cdk/lib/index.js:383:147507)
at async exec4 (/Users/USER/Downloads/MineCloud-1.1.0/node_modules/aws-cdk/lib/index.js:438:51799)
Stack Deployments Failed: Error: The stack named MineCloud failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Received response status [FAILED] from custom resource. Message returned: Error: 2023-05-07T10:38:04.485Z c0c2fd89-6da0-4857-a322-43bba04d23b8 Task timed out after 3.01 seconds
Logs: /aws/lambda/MineCloud-MineClouddiscordcommandsregisterlambda95-n8pTPd6Hw3t2
at invokeUserFunction (/var/task/framework.js:2:6)
at processTicksAndRejections (internal/process/task_queues.js:95:5)
at async onEvent (/var/task/framework.js:1:369)
at async Runtime.handler (/var/task/cfn-response.js:1:1573) (RequestId: cad00625-ab3f-4c88-b864-ee8ec4fd5dc5)
$ %
Timestamp | Logical ID | Status | Status reason |
---|---|---|---|
2023-05-07 16:08:06 UTC+0530 | MineCloud | ROLLBACK_IN_PROGRESS | The following resource(s) failed to create [MineClouddiscordcommandsregisterresource, MineCloudec2instanceInstanceProfile7C956219]. Rollback requested by user. |
2023-05-07 16:08:06 UTC+0530 | MineCloudec2instanceInstanceProfile7C956219 | CREATE_FAILED | Resource creation cancelled |
2023-05-07 16:08:05 UTC+0530 | MineClouddiscordcommandsregisterresource | CREATE_FAILED | Received response status [FAILED] from custom resource. Message returned: Error: 2023-05-07T10:38:04.485Z c0c2fd89-6da0-4857-a322-43bba04d23b8 Task timed out after 3.01 seconds Logs: /aws/lambda/MineCloud-MineClouddiscordcommandsregisterlambda95-n8pTPd6Hw3t2 at invokeUserFunction (/var/task/framework.js:2:6) at processTicksAndRejections (internal/process/task_queues.js:95:5) at async onEvent (/var/task/framework.js:1:369) at async Runtime.handler (/var/task/cfn-response.js:1:1573) (RequestId: cad00625-ab3f-4c88-b864-ee8ec4fd5dc5) |
We can send message to discord channel just using a webhook link:
https://support.discord.com/hc/en-us/articles/228383668-Intro-to-Webhooks
Prepare a Linux script so later we can use it to send update to the discord channel
It will be nice if we can make this even easier for beginner.
Figured out the details about how the backup mechanism should work.
Some possible options are:
Is your feature request related to a problem? Please describe.
Setting up a Minecraft server often requires some properties to be set in multiple files.
For example my kids like to play with "allow-cheats=true" and I want the whitelisting to be enabled on the server so that I know that only my kids an d their friends can access it.
Describe the solution you'd like
I do not have a good solution yet, but I would like to kickoff a discussion about this.
I could imagine that there is a folder where the following files (in the case of a minecraft server) can be put into:
The issue is that I do not want those files to be committed accidentally with all my kids UUIDs and usernames in there. So I could imagine again an approach where we put dummy files in there and one would need to copy them, remove a suffix like "template" and then they will be recognized during server-init and copied to the new machine.
Write CDK code to configure server auto shutdown script.
Please refer to our existing script for checking user connection:
https://gist.github.com/314pies/ac0aa3aa4e42e83363b6ca1a6c426564
This script will auto check for TCP and SSH connection and shut down the server if no connection is found
Make sure the server shutdown when no one is online.
Update Readme if needed.
When something went wrong with our script/service, we want a persistence place to see for logs.
Research and configure the EC2 instance so that we have a centralize folder to look for our logs.
Or at least offer a clear instruction to where we can find the logs.
Make sure the logs exist in the target folder/make sure the instruction is clear and accurate
Update Readme if needed.
Deploy to your dev stack and confirm the message showed up in the Discord channel when running the script.
Document
N/A
We can get Discord text channel webhook by => https://support.discord.com/hc/en-us/articles/228383668-Intro-to-Webhooks
We can send text message to discord channel by a simple curl => https://stackoverflow.com/questions/62227905/using-curl-in-the-cmd-to-send-a-discord-webhook
Combine these 2 together so that every time when the server start/stop, it will send a message to the channel.
Deploy to your dev stack and confirm the message showed up in the Discord channel.
Document
N/A
Having a Minecraft hosting solution is nice. However, we should be able to go beyond that by making this solution generic for other multi-player/socket-based/on-demand server hosting.
Plan out tasks to turn this into a generic hosting solution.
N/A
To reduce our reliance on Discord for operational tasks, we have decided to introduce a web page that allows authorized users to manage game servers. Therefore, we need to implement a Web Application, and we have chosen React.js + Next.js as the development options to ensure scalability and flexibility for future development.
As a result, in this mission, we will outline the prerequisites for this change and use NX to convert the project into a monorepo.
Monorepo Conversion using NX:
Hi! Thanks for the project, it seems interesting.
I just installed MineCloud and it went well. I started it, connected to it, played for a few minutes and then started a backup.
Then I tried to ask for the download link. I'm unfamiliar with S3, so I wont paste the whole answer here in case it contains sensitive information, but it was a long-ass link. When I clicked on it, I got the following error:
InvalidRequestThe authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.[...]
(the [...] is again a chain of character I'm not putting here, don't know if it's security sensitive)
I googled the error, it seems to be a problem with my region, which is eu-west-3
. Looks like US regions support an older authorization mechanism that is not supported in Europe: Stack Overlow exemple
Again thanks a lot ;)
Research and write CDK code that are able to configure Minecraft server as a system service after the EC2 instance start.
Make sure the you are able to connect to Minecraft server after the instance start
Update Readme if needed.
This reddit comment point it out S3 free tier is only for first 12 months.
We should update the readme to make this clear.
It's also a good idea to add a section for creating a budget for AWS accounts.
I'm playing around with cdk deploy and cdk destroy to test the DNS name feature I requested in #68
The "cdk destroy" operation takes ages until it finally fails. I'm not 100% sure yet, but it seems as if the spot instance requests issue, you described somewhere in the comments, led to the issue that a second VM has been created, which does not get deleted by the cdk destroy operation and an attempt to delete the whole stack fails because the ENI is still in use by the second spot instance that is running.
aws ec2 describe-spot-instance-requests --query "SpotInstanceRequests[].SpotInstanceRequestId"
The question is whether there is a way to avoid a second VM to be started, becoming somehow detached from the CDK/CFN state?
Right now, we are using Amazon Linux 2
as our main EC2 image.
However:
Thus, the main goals for this task are
Ubuntu 22.04 LTS
instance-init.ts
if neededHere are some useful resources that can be helpful:
Plan out more tasks when have time.
(Feel free to add a new task if you got any idea)
Right now, our built-in backup feature only backup the /opt/minecloud/server
folder. However, since some games might have different files locations, it will be essential to provide support to backup other folders as well.
Make sure the feature is working with manual testing.
Update Readme/Wiki if needed.
Use tcpdump to check for active connections on UDP.
As the first steps, please set the DEV environment for CDK :)
Resource link: link
Is your feature request related to a problem? Please describe.
Sort of - it's a non issue, really. I don't exactly know what permissions the access key needs.
Describe the solution you'd like
I'd like in the FAQ for there to be a list of accesses it needs.
Describe alternatives you've considered
I ended up just giving it admin access, which usually isn't a good practice.
Additional context
I'm using this on a IAM user to minimize the amount of root keys I give out.
Thank you for your work on the project - it's quite amazing!
Is your feature request related to a problem? Please describe.
Satisfactory supports dedicated servers, but is not supported by MineCloud
Describe the solution you'd like
Add support for Satisfactory.
Describe alternatives you've considered
Completely separate package to run that game, but there should be a lot of shared code with this package. So would make sense just to add it on.
Additional context
Game Website: https://www.satisfactorygame.com/
I would like to work on this feature personally
We wanna a cleaner project and more configuration option.
Write CDK code for creating a new EC2 instance stack. The CDK stack should at least support the following:
region
Make sure the stack deployed succeed.
Update Readme if needed.
We want to setup Discord BOT command when spinning up the stack (this only need to be done once)
This is the tutorial (https://discordjs.guide/creating-your-bot/command-deployment.html#guild-commands) for setting it up with a Node project.
Since it's a Node project, it's likely that we can convert it to a lambda and just treat is as a Custom resource.
(Or use the official API instead https://discord.com/developers/docs/interactions/application-commands#registering-a-command)
N/A
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.