dan-v / awslambdaproxy Goto Github PK
View Code? Open in Web Editor NEWAn AWS Lambda powered HTTP/SOCKS web proxy
License: MIT License
An AWS Lambda powered HTTP/SOCKS web proxy
License: MIT License
I'm trying to run during startup. What does this mean?
2020/04/25 06:36:05 gost.go:657: auto://:8080 on [::]:8080
2020/04/25 06:36:06 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:07 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:08 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:09 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:10 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:11 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:12 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:13 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:14 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:15 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:16 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:17 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:18 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:19 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:20 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:21 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:22 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:23 tunnelconnection.go:226: Waiting for tunnel to be established..
2020/04/25 06:36:24 tunnelconnection.go:226: Waiting for tunnel to be established..
Hi, Thanks very much for this, which is pretty much exactly what I was looking for.
I am trying to reliably run lambda proxies with as much diversity as possible. However, I find that with a fairly long list of regions there is always one or more regions which fail to start (not always the same ones). If one fails, the whole startup fails.
It would be really great if there was an option to ignore any errors, as long as at least one region had started. Ideally with a clear and unambiguous summary of which were set up once the startup has completed.
I have tried dropping the least reliable regions but the list of regions which have never failed to start is getting quite short now :-(
Of course, a dynamic approach, with attempts to restart failed regions and tolerate all errors as long as at least one proxy is usable, would be ideal but it isn't needed for my use case, at least. I tend to run the service for a day or so and am more concerned about reliable startup, as diversely as possible, than with long term usage.
Right now there aren't any instructions in the README.
It seems like terraform does not require
experiments = [variable_validation]
at the first line of variables from newer versions.
Maybe it's just me using the new version?
Secondly
Terraform was able to SSL into the instance, but after that, it ended with
aws_instance.this (remote-exec): Redirecting to /bin/systemctl start docker.service
aws_instance.this: Creation complete after 58s [id=i-0e5499b175a1bd304]
aws_eip_association.this: Creating...
aws_eip_association.this: Creation complete after 1s [id=eipassoc-06475599cbfbc3c6a]
null_resource.start_awslambdaproxy: Creating...
null_resource.start_awslambdaproxy: Provisioning with 'remote-exec'...
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
null_resource.start_awslambdaproxy (remote-exec): (output suppressed due to sensitive value in config)
Error: error executing "/tmp/terraform_1852503017.sh": Process exited with status 127
I have no idea what is happening because the log is blocked, but assuming this is after "start docker.service",
there is something wrong with the docker setups?
tested by default settings given in repo.
Thanks btw, wish to contribute, but I don't know jack s*** about terraform.
Hello I made a Docker image for awslambdaproxy with an OpenSSH server built in: https://github.com/unixfox/awslambdaproxy-docker
Would you like to integrate it into the official awslambdaproxy repository?
What is the username and the password that the proxy server asks for after setting up everything?
Hello, trying to install via Terraform but hitting The maximum number of rules per security group has been reached.
I've increased my Inbound or outbound rules per security group
rules from 60 to 120 but still hitting the limit, is this the correct quota to be adjusting here?
╷
│ Error: error updating Security Group (sg-059c85e095c49e44a): error authorizing Security Group (ingress) rules: RulesPerSecurityGroupLimitExceeded: The maximum number of rules per security group has been reached.
│ status code: 400, request id: a6c1c40f-23ae-423f-a67b-f0a90161b43e
│
│ with aws_security_group.this["us-east-1"],
│ on main.tf line 157, in resource "aws_security_group" "this":
│ 157: resource "aws_security_group" "this" {
│
╵
╷
│ Error: error updating Security Group (sg-0d8c094457070a035): error authorizing Security Group (ingress) rules: RulesPerSecurityGroupLimitExceeded: The maximum number of rules per security group has been reached.
│ status code: 400, request id: 1b7ed22a-9d3e-48cb-95a2-437242a4cdcd
│
│ with aws_security_group.this["us-west-2"],
│ on main.tf line 157, in resource "aws_security_group" "this":
│ 157: resource "aws_security_group" "this" {
│
╵
-sh-4.2$ ./awslambdaproxy setup
Failed to run setup for awslambdaproxy: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors-sh-4.2$ ^C
Hey, can we have a different IP for each request?
When I test the socks proxy using curl, I see it randomly fails:
curl -x socks5://user:password@localhost:8080 https://google.com -v
* Trying 127.0.0.1:8080...
* TCP_NODELAY set
* SOCKS5 communication to google.com:443
* SOCKS5 connect to IPv4 *.*.*.*:443 (locally resolved)
* Can't complete SOCKS5 connection to *.*.*.*:443. (4)
* Closing connection 0
curl: (7) Can't complete SOCKS5 connection to *.*.*.*:443. (4)
Expected behavior: [What you expected to happen]
No failure
Actual behavior: [What actually happened]
Connection errors
./awslambdaproxy --version
awslambdaproxy version 0.0.14
./awslambdaproxy run -r us-west-2,us-west-1,us-east-1,us-east-2,ca-central-1,eu-central-1,eu-west-1,eu-west-2,eu-west-3 -f 360s
2020/11/21 15:12:42 tunnelconnection.go:125: 2 Unique Lambda IPs used so far
2020/11/21 15:12:42 tunnelconnection.go:126: ---------------
2020/11/21 15:13:01 tunnelconnection.go:196: Delayed cleanup now running for 127.0.0.1:57746
2020/11/21 15:13:01 tunnelconnection.go:200: Removing tunnel 127.0.0.1:57746
2020/11/21 15:13:01 tunnelconnection.go:171: error closing connection for connectionID=127.0.0.1:57746: close tcp 127.0.0.1:8081->127.0.0.1:57746: use of closed network connection
2020/11/21 15:18:40 lambdaexecution.go:54: Executing Lambda function in region us-east-1
2020/11/21 15:18:40 lambdaexecution.go:65: Setting invoke configuration maximumRetryAttempts=0 maximumEventAgeInSeconds=1800
2020/11/21 15:18:40 lambdaexecution.go:95: Invoking Lambda function with UUID=***
2020/11/21 15:18:41 tunnelconnection.go:81: Accepted tunnel connection from 127.0.0.1:38564
2020/11/21 15:18:41 tunnelconnection.go:89: Established session to 127.0.0.1:38564
2020/11/21 15:18:41 tunnelconnection.go:192: Tunnel '127.0.0.1:33912' that is being closed still has 916 open streams. Delaying cleanup for 20s.
2020/11/21 15:18:42 tunnelconnection.go:110: ---------------
2020/11/21 15:18:42 tunnelconnection.go:111: Current Lambda IP Address: *.*.*.*
2020/11/21 15:18:42 tunnelconnection.go:112: Active Lambda tunnel count: 2
2020/11/21 15:18:42 tunnelconnection.go:115: Lambda Tunnel #1
2020/11/21 15:18:42 tunnelconnection.go:116: Connection ID: 127.0.0.1:38564
2020/11/21 15:18:42 tunnelconnection.go:117: Start Time: 2020-11-21T15:18:41
2020/11/21 15:18:42 tunnelconnection.go:118: Active Streams: 1
2020/11/21 15:18:42 tunnelconnection.go:115: Lambda Tunnel #2
2020/11/21 15:18:42 tunnelconnection.go:116: Connection ID: 127.0.0.1:33912
2020/11/21 15:18:42 tunnelconnection.go:117: Start Time: 2020-11-21T15:12:41
2020/11/21 15:18:42 tunnelconnection.go:118: Active Streams: 916
2020/11/21 15:18:42 tunnelconnection.go:125: 3 Unique Lambda IPs used so far
2020/11/21 15:18:42 tunnelconnection.go:126: ---------------
2020/11/21 15:19:01 tunnelconnection.go:196: Delayed cleanup now running for 127.0.0.1:33912
2020/11/21 15:19:01 tunnelconnection.go:200: Removing tunnel 127.0.0.1:33912
2020/11/21 15:19:01 tunnelconnection.go:171: error closing connection for connectionID=127.0.0.1:33912: close tcp 127.0.0.1:8081->127.0.0.1:33912: use of closed network connection
2020/11/21 15:24:40 lambdaexecution.go:54: Executing Lambda function in region us-east-2
2020/11/21 15:24:40 lambdaexecution.go:65: Setting invoke configuration maximumRetryAttempts=0 maximumEventAgeInSeconds=1800
2020/11/21 15:24:40 lambdaexecution.go:95: Invoking Lambda function with UUID=***
2020/11/21 15:24:41 tunnelconnection.go:192: Tunnel '127.0.0.1:38564' that is being closed still has 872 open streams. Delaying cleanup for 20s.
2020/11/21 15:24:42 tunnelconnection.go:81: Accepted tunnel connection from 127.0.0.1:42924
2020/11/21 15:24:42 tunnelconnection.go:89: Established session to 127.0.0.1:42924
2020/11/21 15:24:42 tunnelconnection.go:110: ---------------
2020/11/21 15:24:42 tunnelconnection.go:111: Current Lambda IP Address: 18.191.212.184
2020/11/21 15:24:42 tunnelconnection.go:112: Active Lambda tunnel count: 2
2020/11/21 15:24:42 tunnelconnection.go:115: Lambda Tunnel #1
2020/11/21 15:24:42 tunnelconnection.go:116: Connection ID: 127.0.0.1:42924
2020/11/21 15:24:42 tunnelconnection.go:117: Start Time: 2020-11-21T15:24:42
2020/11/21 15:24:42 tunnelconnection.go:118: Active Streams: 2
2020/11/21 15:24:42 tunnelconnection.go:115: Lambda Tunnel #2
2020/11/21 15:24:42 tunnelconnection.go:116: Connection ID: 127.0.0.1:38564
2020/11/21 15:24:42 tunnelconnection.go:117: Start Time: 2020-11-21T15:18:41
2020/11/21 15:24:42 tunnelconnection.go:118: Active Streams: 872
2020/11/21 15:24:42 tunnelconnection.go:125: 4 Unique Lambda IPs used so far
2020/11/21 15:24:42 tunnelconnection.go:126: ---------------
2020/11/21 15:25:01 tunnelconnection.go:49: Failed to accept user connection
2020/11/21 15:25:01 tunnelconnection.go:196: Delayed cleanup now running for 127.0.0.1:38564
2020/11/21 15:25:01 tunnelconnection.go:200: Removing tunnel 127.0.0.1:38564
2020/11/21 15:25:01 tunnelconnection.go:171: error closing connection for connectionID=127.0.0.1:38564: close tcp 127.0.0.1:8081->127.0.0.1:38564: use of closed network connection
[bidirectionalCopy has no timeout]
Expected behavior: [What you expected to happen]
Actual behavior: [What actually happened]
./awslambdaproxy version
awslambdaproxy version 0.0.12
./awslambdaproxy -r us-west-2,us-west-1 -f 60
docker run -it --rm --entrypoint /app/awslambdaproxy vdan/awslambdaproxy -v
awslambdaproxy version 0.0.12
docker run -d vdan/awslambdaproxy -r us-west-2,us-west-1 -f 60
func waitTimeout(wg *sync.WaitGroup, timeout time.Duration) bool {
done := make(chan struct{})
go func() {
wg.Wait()
close(done)
}()
select {
case <-done:
return false
case <-time.After(timeout):
return true
}
}
func bidirectionalCopy(src io.ReadWriteCloser, dst io.ReadWriteCloser, timeout_S float64) bool {
defer dst.Close()
defer src.Close()
var wg sync.WaitGroup
wg.Add(1)
go func() {
io.Copy(dst, src)
wg.Done()
}()
wg.Add(1)
go func() {
io.Copy(src, dst)
wg.Done()
}()
//wg.Wait()
return !waitTimeout(&wg, time.Duration(timeout_S)*time.Second)
}
I have some version of this project on a server. I noticed it created 6000 .ssh/authorized_keys. I don't know how to check the version, the command prompt doesn't seem to give me that info. I also installed version 13 on a new instance. So far it created 2 keys.
Probably running with low -f param or running the server under supervisor and periodically restart the daemon increase the amount of left over keys.
Expected behavior: I guess this package should safely clear the keys after it is done with them.
Actual behavior: Over time authorized_keys will get huge.
# version 13
awslambdaproxy run -r us-west-2,us-west-1,us-east-1,us-east-2,ca-central-1,eu-central-1,eu-west-1,eu-west-2,eu-west-3 -f 360s
When I run:
awslambdaproxy run us-west-1,us-west-2,us-east-1,us-east-2
Getting:
Must specify environment variable AWS_ACCESS_KEY_ID
The EC2 instance should already have the policies needed via the attachd IAM role.
Describe the solution you'd like
I want to have a single PC but running multiple tasks with different IP addresses.
In this case, I will connect to lambda functions and all the request maintains the same IP until after a certain amount of time.
Describe alternatives you've considered
Perhaps running ./awslambdaproxy multiple times? However, it will still go through the ec2 instance.
I sent recursive 100 parallel requests and got panic error in 10 seconds.
./awslambdaproxy run -r us-west-2,us-west-1,us-east-1,us-east-2 -l "cL3Rjea6E:uQ06OWgdr@:8080" -m 512 -f 10m --debug
)Expected behavior: [What you expected to happen]
I expect, that awslmbdaproxy will work
Actual behavior: [What actually happened]
I got panic error
awslambdaproxy 0.0.13 and 0.0.14
Linux service 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1 (2020-01-26) x86_64 GNU/Linux
Error log is too big. I've attached it.
awslambdaproxy.log
Excellent concept! I think it's a great idea.
Within 15 mins I kept on getting disconnected with error ERR_TUNNEL_CONNECTION_FAILED
[Description of the issue]
Expected behavior: [What you expected to happen]
2020/10/26 09:57:19 tunnelconnection.go:81: Accepted tunnel connection from 127.0.0.1:39214
2020/10/26 09:57:19 tunnelconnection.go:89: Established session to 127.0.0.1:39214
2020/10/26 09:57:20 tunnelconnection.go:110: ---------------
2020/10/26 09:57:20 tunnelconnection.go:111: Current Lambda IP Address: 13.229.136.52
2020/10/26 09:57:20 tunnelconnection.go:112: Active Lambda tunnel count: 1
2020/10/26 09:57:20 tunnelconnection.go:115: Lambda Tunnel #1
2020/10/26 09:57:20 tunnelconnection.go:116: Connection ID: 127.0.0.1:39214
2020/10/26 09:57:20 tunnelconnection.go:117: Start Time: 2020-10-26T09:57:19
2020/10/26 09:57:20 tunnelconnection.go:118: Active Streams: 1
2020/10/26 09:57:20 tunnelconnection.go:125: 1 Unique Lambda IPs used so far
2020/10/26 09:57:20 tunnelconnection.go:126: ---------------
Actual behavior: [What actually happened]
2020/10/26 10:04:34 tunnelconnection.go:235: Waiting for tunnel to be established..
2020/10/26 10:04:35 tunnelconnection.go:235: Waiting for tunnel to be established..
2020/10/26 10:04:36 tunnelconnection.go:235: Waiting for tunnel to be established..
2020/10/26 10:04:37 tunnelconnection.go:235: Waiting for tunnel to be established..
2020/10/26 10:04:38 tunnelconnection.go:235: Waiting for tunnel to be established..
2020/10/26 10:04:39 tunnelconnection.go:235: Waiting for tunnel to be established..
2020/10/26 10:04:40 tunnelconnection.go:235: Waiting for tunnel to be established..
./awslambdaproxy -v
awslambdaproxy version 0.0.14
./awslambdaproxy run -r ap-southeast-1
INFO[0000] server has been configured with the following values bypass= debug=false lambdaExecutionFrequency=14m20s lambdaIamRole=awslambdaproxy-role lambdaMemory=128 lambdaName=awslambdaproxy lambdaRegions="[ap-southeast-1]" lambdaTimeoutSeconds=890 proxyDebug=false proxyListeners="[admin:awslambdaproxy@:8080]" publicIPClient="http://checkip.amazonaws.com/" reverseTunnelSSHPort=22 reverseTunnelSSHUser=ec2-user
INFO[0000] setting up lambda infrastructure
2020/10/26 10:06:18 infrastructure.go:115: Setting up Lambda function in region: ap-southeast-1
INFO[0002] starting ssh tunnel manager
2020/10/26 10:06:19 ssh.go:93: Generated SSH key: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvHO88vzF/xIgGG2B4ZCQYFQAGuz4A36IWRWpGOWcfYBwTBShxDD0aURs6yCMkFM3lPGEQ+4QTE5bvHA6n3X4RmIJMa449Q3PXdj6eMlJnm6/M1aJuftMyhjv9nuvS7we4yHWRkESEusxiXIMz/wqfmsPNyT76sZlf7lqL1zvENhe8/qCdQ9BVtTvwpvUpW0DkzRD3B4auGg0osQc8HTQ4NvfgzBDzlw(*deleted some of it)
2020/10/26 10:06:19 ssh.go:99: Added entry to authorized keys file /home/ec2-user/.ssh/authorized_keys
INFO[0002] starting local proxy
INFO[0002] starting connection manager
2020/10/26 10:06:19 tunnelconnection.go:291: Started user listener on port 8082
2020/10/26 10:06:19 tunnelconnection.go:280: Started tunnel listener on port 8081
INFO[0002] starting lambda execution manager
INFO[0002] #######################################
INFO[0002] proxy ip address: 54.169.124.179
INFO[0002] listeners: [admin:awslambdaproxy@:8080]
INFO[0002] #######################################
2020/10/26 10:06:19 lambdaexecution.go:34: Using public IP 54.169.124.179
2020/10/26 10:06:19 lambdaexecution.go:35: Lambda execution frequency 14m20s
2020/10/26 10:06:19 lambdaexecution.go:54: Executing Lambda function in region ap-southeast-1
2020/10/26 10:06:19 lambdaexecution.go:65: Setting invoke configuration maximumRetryAttempts=0 maximumEventAgeInSeconds=1800
2020/10/26 10:06:19 lambdaexecution.go:95: Invoking Lambda function with UUID=e4dc9ae5-1772-11eb-becb-06e7a13d5fbc
2020/10/26 10:06:20 gost.go:658: auto://:8080 on [::]:8080
2020/10/26 10:06:20 tunnelconnection.go:235: Waiting for tunnel to be established..
2020/10/26 10:06:21 tunnelconnection.go:235: Waiting for tunnel to be established..
2020/10/26 10:06:22 tunnelconnection.go:235: Waiting for tunnel to be established..
2020/10/26 10:06:23 tunnelconnection.go:235: Waiting for tunnel to be established..
2020/10/26 10:06:24 tunnelconnection.go:235: Waiting for tunnel to be established..
2020/10/26 10:06:25 tunnelconnection.go:235: Waiting for tunnel to be established..
2020/10/26 10:06:26 tunnelconnection.go:235: Waiting for tunnel to be established..
im not fairly good with aws. but as i know for now.
i created an ec2 on a default vpc and run the command like readme it is working fine and i also got the result like what expected to happen
at the top
but when i try to create an ec2 on another vpc it would not work
even when i have already open port 22 and 8080
This error has always been occurring, but it just happens to be logged now:
error closing connection for connectionID=127.0.0.1:47858: close tcp 127.0.0.1:8081->127.0.0.1:47858: use of closed network connection
Need to track down root cause of slow loading time specifically for reddit.com. Can take up to 10s to load, while other websites do not see this behavior.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.