Auto Maple is an intelligent Python program that plays MapleStory, a 2D side-scrolling MMORPG, using simulated key presses, TensorFlow machine learning, OpenCV template matching, and other computer vision techniques.
Auto Maple uses OpenCV template matching to determine the bounds of the minimap as well as the various elements within it, allowing it to accurately track the player's in-game position. If record_layout is set to True, Auto Maple will record the player's previous positions in a quadtree-based Layout object, which is periodically saved to a file in the "layouts" directory. Every time a new routine is loaded, its corresponding layout file, if it exists, will also be loaded. This Layout object uses the A* search algorithm on its stored points to calculate the shortest path from the player to any target location, which can dramatically improve the accuracy and speed at which routines are executed.
|
Click here to view the above routine. |
The above video shows Auto Maple consistently performing a mechanically advanced ability combination.
| Designed with modularity in mind, Auto Maple can operate any character in the game as long as it is provided with a list of in-game actions, or a "command book". A command book is a Python file that contains multiple classes, one for each in-game ability, that tells the program what keys it should press and when to press them. Once a command book is imported, its classes are automatically compiled into a dictionary that Auto Maple can then use to interpret commands within routines. Commands have access to all of Auto Maple's global variables, which can allow them to actively change their behavior based on the player's position and the state of the game. |
|
|
A routine is a user-created CSV file that tells Auto Maple where to move and what commands to use at each location. A custom compiler within Auto Maple parses through the selected routine and converts it into a list of Component objects that can then be executed by the program. An error message is printed for every line that contains invalid parameters, and those lines are ignored during the conversion.
Below is a summary of the most commonly used routine components:
|
| Auto Maple has the ability to automatically solve "runes", or in-game arrow key puzzles. It first uses OpenCV's color filtration and Canny edge detection algorithms to isolate the arrow keys and reduce as much background noise as possible. Then, it runs multiple inferences on the preprocessed frames using a custom-trained TensorFlow model until two inferences agree. Because of this preprocessing, Auto Maple is extremely accurate at solving runes in all kinds of (often colorful and chaotic) environments. |
Click below to see Auto Maple in action:
- Download and install Python3.
- Download and install the latest version of CUDA Toolkit.
- Download and unzip the latest Auto Maple release.
- Download the TensorFlow model and unzip the "models" folder into Auto Maple's "assets" directory.
-
Inside Auto Maple's main directory, open a command prompt and run:
python -m pip install -r requirements.txt -
Lastly, create a desktop shortcut by running
python setup.py. This shortcut uses absolute paths, so feel free to move it wherever you want. However, if you move Auto Maple's main directory, you will need to runpython setup.pyagain to generate a new shortcut.
From working on this project, I truly learned a lot. Auto Maple not only allowed me to apply the knowledge I learned in lectures to a challenging problem, but it also introduced me to many exciting and complex concepts such as machine learning and Canny edge detection. However, perhaps more importantly, working on Auto Maple has given me a deeper appreciation of human problem solving and a clearer understanding of just how hard it is for a computer to emulate that.
Early in this project, when I was still trying to get the character to move to locations on the minimap, I resorted to letting the program blindly move horizontally and vertically until it reached its target. This was very inefficient and often resulted in the character getting stuck. Later, I realized that memory was the main aspect Auto Maple was missing: human gamers remember where they've been, which places are safe to walk on, and which places aren't. This gave me the idea to create a Layout class to help the program chart a path in advance based on where it has already been, much like a human.
In the end, Auto Maple is still far from perfect. It can't predict and prevent missteps like we can. It can't reproduce the fluid actions of a practiced gamer. But acknowledging these shortcomings inspires me to continue learning and experimenting with new ways to make programs faster and smarter.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent