wuba / antenna Goto Github PK

建议：将下方的 max_length 改为255

('value', models.CharField(blank=True, help_text='配置内容', max_length=32, null=True)),

https://github.com/wuba/Antenna/blob/main/modules/config/migrations/0001_initial.py#L20

操作过程：
1、git clone 到本地
2、使用centos默认镜像创建docker
docker run -it -d --name Antenna -v /home/ubuntu/Antenna:/Antenna -p 8000:8000 centos bash
3、安装依赖时发现无法安装
执行install.sh 后，发现没有python环境。
根据下面这个补充yum源问题后安装python3
https://techglimpse.com/failed-metadata-repo-appstream-centos-8/
yum install python3
然后手动去安装依赖包，发现无法安装mysqlclient==2.1.0 这个模块报错
pip3 install -r requirements.txt
或者 pip3 install mysqlclient==2.1.0 报错,错误信息大概如下：

Installing collected packages: mysqlclient, dnslib, djangorestframework-bulk, django-extensions, django-cors-headers
    Running setup.py install for mysqlclient ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3.6 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vk6arqx4/mysqlclient_f83b1bcb1ac24639b767cb668f1111b7/setup.py'"'"'; __file__='"'"'/tmp/pip-install-vk6arqx4/mysqlclient_f83b1bcb1ac24639b767cb668f1111b7/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-pzbssw7n/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.6m/mysqlclient
         cwd: /tmp/pip-install-vk6arqx4/mysqlclient_f83b1bcb1ac24639b767cb668f1111b7/

省略一万字。。。
-std=c99 -m64
    unable to execute 'gcc': No such file or directory
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3.6 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vk6arqx4/mysqlclient_f83b1bcb1ac24639b767cb668f1111b7/setup.py'"'"'; __file__='"'"'/tmp/pip-install-vk6arqx4/mysqlclient_f83b1bcb1ac24639b767cb668f1111b7/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-pzbssw7n/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.6m/mysqlclient Check the logs for full command output.

存在消息记录，OpenAPI查询结果却为空

这个项目目前还有维护吗？博客已经无法访问了。

1、支持https服务，方便兼容一些业务场景
2、记录完整请求头，方便测试发现外带请求头中泄露的非常规字段的敏感信息。比如：
Token：xxxx

任务管理-新建-500报错

新增设置，可自定义几天清理一次数据。
因为很多时候数据都是ssrf或测试数据，不需要长期保留，所以有定时清理的话，会清晰一些

目前Antenna在dockerhub的镜像仅支持amd64架构,建议构建Multi-Architecture镜像.

可以单独打tag:

docker build --pull --platform=linux/arm64 -f Dockerfile -t jihongjun/antenna:v1.3.5-arm64 .

或者使用docker buildx创建多架构镜像

例如接口：
http://baidu.hun/api/v1/messages/manage/api/?apikey=xxxx&uri=&task__name=&message_type=&domain=&content=&page=1&page_size=10

测试发现domain 参数不支持模糊查询，在DNSLog场景中，需要提供完整的domain才能查询回数据。非常不友好！

期望：
API可查询的字段都实现不完全匹配。即：输入一部分字符串，查询包含它的所有记录，按照时间由近到远顺序返回。

root@MacBook-Pro-M2 ~ % dig blog.antenna.cool @8.8.8.8

; <<>> DiG 9.10.6 <<>> blog.antenna.cool @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36027
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;blog.antenna.cool. IN A

;; ANSWER SECTION:
blog.antenna.cool. 600 IN A 8.219.104.156

;; Query time: 600 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sat Jul 29 17:39:59 CST 2023
;; MSG SIZE rcvd: 62

user@MacBook-Pro-M2 ~ %

修改组件配置内容后提交保存，前端js无报错但是也没有网络请求

使用最新的docker镜像运行一段时间以后，dns解析失败并报错，报错内容：
匹配域名 xxxxxx 匹配结果: False
匹配域名 *.xxxxxxx 匹配结果: True
请求解析域名： a1.xxxxxxx
Unhandled Error
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/twisted/names/server.py", line 538, in messageReceived
self.handleQuery(message, proto, address)
File "/Antenna/modules/template/depend/listen/dnslog.py", line 43, in handleQuery
return server.DNSServerFactory.handleQuery(self, message, protocol, address)
File "/usr/lib/python3.10/site-packages/twisted/names/server.py", line 371, in handleQuery
self.resolver.query(query)
File "/usr/lib/python3.10/site-packages/twisted/names/common.py", line 78, in query
return defer.maybeDeferred(method, query.name.name, timeout)
--- ---
File "/usr/lib/python3.10/site-packages/twisted/internet/defer.py", line 167, in maybeDeferred
result = f(*args, **kw)
File "/usr/lib/python3.10/site-packages/twisted/names/common.py", line 84, in lookupAddress
return self._lookup(dns.domainString(name), dns.IN, dns.A, timeout)
File "/usr/lib/python3.10/site-packages/twisted/names/resolve.py", line 76, in _lookup
d = self.resolvers[0].query(q, timeout)
File "/Antenna/modules/template/depend/listen/dnslog.py", line 122, in query
return defer.succeed(self._doDynamicResponse(query))
File "/Antenna/modules/template/depend/listen/dnslog.py", line 95, in _doDynamicResponse
if len(list(self.dns_recoed.get(domain=domain.lower()).value)) == 1:
File "/usr/lib/python3.10/site-packages/django/db/models/query.py", line 431, in get
num = len(clone)
File "/usr/lib/python3.10/site-packages/django/db/models/query.py", line 262, in len
self._fetch_all()
File "/usr/lib/python3.10/site-packages/django/db/models/query.py", line 1324, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/lib/python3.10/site-packages/django/db/models/query.py", line 51, in iter
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/usr/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql
cursor.execute(sql, params)
File "/usr/lib/python3.10/site-packages/django/db/backends/utils.py", line 66, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/usr/lib/python3.10/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/lib/python3.10/site-packages/django/db/backends/utils.py", line 79, in _execute
with self.db.wrap_database_errors:
File "/usr/lib/python3.10/site-packages/django/db/utils.py", line 90, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 73, in execute
return self.cursor.execute(query, args)
File "/usr/lib/python3.10/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/usr/lib/python3.10/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/usr/lib/python3.10/site-packages/pymysql/connections.py", line 547, in query
self._execute_command(COMMAND.COM_QUERY, sql)
File "/usr/lib/python3.10/site-packages/pymysql/connections.py", line 793, in _execute_command
raise err.InterfaceError(0, "")
django.db.utils.InterfaceError: (0, '')

twisted 貌似在处理较大的请求体时会将http header 和body分开处理，导致传入的request body会使程序出错

请求体不大时正常处理

dockerfile里面设定下时区,否则时间有差异.

最新版 Antenna 启动后一段时间报Processing Failed
查看antenna_https_stdout.log日志显示

Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/twisted/protocols/basic.py", line 548, in dataReceived
why = self.lineReceived(line)
File "/usr/lib/python3.10/site-packages/twisted/web/http.py", line 2155, in lineReceived
self.allContentReceived()
File "/usr/lib/python3.10/site-packages/twisted/web/http.py", line 2281, in allContentReceived
req.requestReceived(command, path, version)
File "/usr/lib/python3.10/site-packages/twisted/web/http.py", line 1005, in requestReceived
self.process()
--- ---
File "/usr/lib/python3.10/site-packages/twisted/web/server.py", line 229, in process
self.render(resrc)
File "/usr/lib/python3.10/site-packages/twisted/web/server.py", line 294, in render
body = resrc.render(self)
File "/Antenna/modules/template/depend/listen/httpslog.py", line 67, in render
task_config_item = TaskConfigItem.objects.filter(task_config__key=self.key, task__status=1).first()
File "/usr/lib/python3.10/site-packages/django/db/models/query.py", line 674, in first
for obj in (self if self.ordered else self.order_by('pk'))[:1]:
File "/usr/lib/python3.10/site-packages/django/db/models/query.py", line 280, in iter
self._fetch_all()
File "/usr/lib/python3.10/site-packages/django/db/models/query.py", line 1324, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/lib/python3.10/site-packages/django/db/models/query.py", line 51, in iter
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/usr/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql
cursor.execute(sql, params)
File "/usr/lib/python3.10/site-packages/django/db/backends/utils.py", line 66, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/usr/lib/python3.10/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/lib/python3.10/site-packages/django/db/backends/utils.py", line 79, in _execute
with self.db.wrap_database_errors:
File "/usr/lib/python3.10/site-packages/django/db/utils.py", line 90, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 73, in execute
return self.cursor.execute(query, args)
File "/usr/lib/python3.10/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/usr/lib/python3.10/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/usr/lib/python3.10/site-packages/pymysql/connections.py", line 547, in query
self._execute_command(COMMAND.COM_QUERY, sql)
File "/usr/lib/python3.10/site-packages/pymysql/connections.py", line 793, in _execute_command
raise err.InterfaceError(0, "")
django.db.utils.InterfaceError: (0, '')

增加DNS重绑定

环境

• 阿里云
• CentOS Linux release 7.7.1908 (Core)

问题

启动报错，如下

这是因为docker内运行脚本的用户权限问题导致的，删除Dockerfile文件的以下三行，删除旧镜像，重新构建即可。

使用网易163邮箱发送邮件失败，前端提示发送邮件失败。

建议后台出个数据备份导入导出的功能方便数据迁移

随意监听一个HTTP组件
复现：

http://xxx.xxx.fun/whoami?=1122222

curl -A o -o- -L -F file=@/etc/hosts http://xxx.xxx.fun/

查看记录，GET无参数记录，只有URI
POST直接返回报错。

{"code":0,"message":"Method "POST" not allowed.","data":{}}

单个域名cloudflare如何配置

建议配置docker-compose.yml 中，防止mysql服务暴露公网

ports:
      - "127.0.0.1:3306:3306"

docker环境api接口查询500报错，查看日志没有详细报错内容

报错信息如下：

[30/Aug/2022 06:18:20] "GET /api/v1/configs/manage/?page_size=20 HTTP/1.1" 200 390
Internal Server Error: /api/v1/auth/sendmail/test/
Traceback (most recent call last):
  File "/home/antenna/.local/lib/python3.10/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/antenna/.local/lib/python3.10/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/home/antenna/.local/lib/python3.10/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/home/antenna/.local/lib/python3.10/site-packages/rest_framework/viewsets.py", line 125, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/antenna/.local/lib/python3.10/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/home/antenna/.local/lib/python3.10/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/home/antenna/.local/lib/python3.10/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/home/antenna/.local/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/antenna/modules/account/views.py", line 57, in test
    serializer.is_valid(raise_exception=True)
  File "/home/antenna/.local/lib/python3.10/site-packages/rest_framework/serializers.py", line 220, in is_valid
    self._validated_data = self.run_validation(self.initial_data)
  File "/home/antenna/.local/lib/python3.10/site-packages/rest_framework/serializers.py", line 422, in run_validation
    value = self.validate(value)
  File "/antenna/modules/account/serializers.py", line 50, in validate
    del attrs['verify_code']
KeyError: 'verify_code'

操作流程同http://blog.antenna.cool/docs/%E5%85%B3%E4%BA%8E%E9%85%8D%E7%BD%AE/DNS
但基于腾讯云自定义DNS存在部分问题

建议增加系统运行日志，在系统出问题的时候，可以定位问题原因