The actor model is a math model for concurrent computations, where the ๐ง actor symbolizes the universal primitive of concurrent computation.
For this project, I will showcase four examples where Lithops (Python-based library) can work using the actor model. Lithops has three execution modes:
- Localhost
- Serverless
- Standalone
This project is a technical demonstration of what you can achieve with serverless functions and how to adapt our code using the actor model.
Table of Contents
The code related to the actor model implementation within Lithops was based on this source code: https://github.com/danielBCN/lithops-actors
The author is Daniel Barcelona: https://github.com/danielBCN
Code: ./actor_model/director.py
Creating a virtual environment, activating it, and installing the necessary packages.
# Create Python's virtual environment
python3 -m venv .venv
# Activate Python's virtual environment
source .venv/bin/activate
# Upgrade pip to the latest version
python -m pip install --upgrade pip
# Production
python -m pip install -r requirements.txt
# Development
# python -m pip install -r requirements-dev.txt
Exit Python's virtual environment.
# Exit Python's virtual environment
deactivate
# Remove Python's virtual environment folder
rm -rf .venv/
- Counter:
python actor_model/counter.py
- Ping pong:
python actor_model/ping_pong.py
- Bank account:
python actor_model/bank_account.py
- Bank account v2:
python actor_model/bank_account_v2.py
- MNIST:
python actor_model/mnist.py
View logs after each run.
lithops logs poll
The datasets belong to:
- Yann LeCun, Courant Institute, NYU
- Corinna Cortes, Google Labs, New York
- Christopher J.C. Burges, Microsoft Research, Redmond
http://yann.lecun.com/exdb/mnist/
The files have been decompressed and stored within ./datasets/
:
- Test set images:
t10k-images-idx3-ubyte
- Test set labels:
t10k-labels-idx1-ubyte
- Training set images:
train-images-idx3-ubyte
- Training set labels:
train-images-idx1-ubyte
The source code for MNIST with NumPy has been adapted from this source: https://github.com/karynaur/mnist-from-numpy/blob/main/mnist.py
The author is Aditya Srinivas Menon: https://github.com/karynaur
https://github.com/lithops-cloud/lithops
Configuration file for serverless mode:
# Configuration file example for Lithops serverless mode
cp .lithops_config.example .lithops_config
Lithops configuration: Serverless mode
Lithops configuration: AWS Lambda
-
The first iteration will use Amazon S3 (Amazon Simple Storage Service) as a cloud object storage.
Lithops configuration: AWS S3
-
The subsequent runs will use Redis, due to Redis Stack nature of offering different data structures.
Lithops configuration: Redis
In my particular case, I'm using DigitalOcean's Redis service as it is relatively easy to spin up a new instance and destroy it afterwards.
The RESP.app documentation gives a wonderful insight of the different solutions provided by different cloud providers: https://docs.resp.app/en/latest/quick-start/
- The main reason against using AWS ElastiCache (Amazon's managed Redis), is that you can only access it within your VPC (Virtual Private Cloud). Instructions are provided to circumvent these restrictions.
- Azure Cache for Redis can be accessed from outside of Azure's VPC but Terraform's integration with Azure is not quite stable for me.
- The port number is usually
6380
instead of the default6379
port.
- The port number is usually
- DigitalOcean uses a different port such as
25061
.
Redis out of the box supports 16 databases, 0 is the default one.
To delete the current database run the following command.
flushdb
Terraform by HashiCorp is used for IaC (Infrastructure as Code). I've included an example for DigitalOcean managed Redis instance.
This Redis instance has ๐ encryption at rest and in transit: https://docs.digitalocean.com/products/databases/redis/how-to/secure/
-
Install Terraform CLI: https://learn.hashicorp.com/tutorials/terraform/install-cli
-
Install Terraform providers.
# Install providers terraform init
-
Create a variable definition file for DigitalOcean's personal access token.
# File with DigitalOcean personal access token ignored by GIT due to .gitignore cat << EOF >> digital-ocean.tfvars digital_ocean_token = "" EOF
-
Use the variable definition file to spin up the Redis instance.
# Use variable definitions and create an output file with the changes needed for Terraform apply terraform plan --var-file=digital-ocean.tfvars --input=false --out lithops.tfplan # DigitalOcean will now spin a managed Redis instance terraform apply "lithops.tfplan"
-
Retrieve Redis configuration.
# After several minutes retrieve host, port, and password terraform show --json
-
Destroy the Redis instance to keep costs low.
# Destroy managed Redis instance terraform destroy --var-file=digital-ocean.tfvars
Documentation: https://www.terraform.io/cli/commands
# Install providers
terraform init
# Format code
terraform fmt .
# Validate Terraform code
terraform validate
# Dry run
terraform plan
# Install
terraform apply
# Uninstall
terraform destroy
# View configuration and piping results to jq (Command-line JSON processor) https://github.com/stedolan/jq
terraform show --json | jq .
Distributed under the MIT License. See LICENSE.txt
for more information.
Michael A. Johnson Lucas - mjohnson(at)uoc.edu
Project Link: https://github.com/ephod/lithops_actor_model