Hi,

please add the locationDestination to the retrieveAlerts JSON (and its related backend logic).

Proposal for naming:

lat -> sourceLat
lng -> sourceLng

+new:
destLat
destLng

Regards,
Aydin

I'd like to implement a stacked-bar chart (like this one http://code.shutterstock.com/rickshaw/examples/extensions.html)

Can you create a new endpoint (business logic based on /alert/retrieveAlertsCount)
which adds also the honeypot-type?

Example json response:

{
"AlertCountTotal": 24,
"AlertCountPerType": {
"ssh": 121,
"honeydingsbums": 12344,
"iskender kebab": 3434,
"kommeinergehtnoch": 543
}
}

I would call it periodically (passing the time variable in the url as known) and
display the data in a chart in near-realtime.

def authenticate(username, token)

[
{
"datasetAlertsPerMonth":{
"20170701":152682,
"20170702":219977,
"20170703":146094
}
}
]

root@ews02:/opt/PEBA# tail -f error.log
raise value
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functionsrule.endpoint
File "/opt/PEBA/webservice.py", line 224, in retrieveIPs
username, password = (getCreds(request.data.decode('utf-8')))
File "/opt/PEBA/webservice.py", line 36, in getCreds
username = root.find("./Authentication/username").text.decode('utf-8')
AttributeError: 'str' object has no attribute 'decode'

Temporary fix: remove the .decode('utf-8')

Hi,

I deployed PEBA on ES6 and put the test data, but I could not put data to PEBA.
I found that there are some differences between "setup-es-indices.py" and "setupES6Indices.sh".

Do I need to update the script to use ES6?

https://github.com/schmalle/ask-elastic-py

[
[
"2016-11",
#monat-tag (mit leading zero) [
32.061707,
118.7778,
3925821,
#lat,
lng,
count 21.033295,
105.850006,
758705,
23.1167,
113.25,
338788,
48.86,
2.350006,
112268,
12.983307,
77.58331,
82639
]
],
[
"2016-10",
[
51,
9,
72552,
59.894394,
30.264206,
68928,
39.928894,
116.388306,
49394,
38,
-97,
48338,
34.099503,
-118.4143,
18730,
47.6801,
-122.120605,
12501,
47,
29,
12254,
37.386,
-122.0838,
12219,
51.533295,
0.69999695,
9321,
-31.942795,
115.8439,
6155
]
]
]

Hi,

I'd like to know how to create statistics index in elasticsearch.
I accessed public get endpoint 'alert/getStats?' on my peba server, I got this error message.

[2019-08-27 07:24:15,157] ERROR in tpotstats: ElasticSearch error: TransportError(404, 'index_not_found_exception', 'no such index')

Thanks,

1. Please filter out those entries having an empty targetCountry-value because I can't display them on a map.

2. The requestString value (which is parameter in the tacho) is empty in very often cases, this makes the live-ticker table looking empty / ugly.

In the old JSON (http://sicherheitstacho.eu/alertsJSON) it had always a value:

,"analyzerType":"Konsole/Shell","requestString":"SSH Honeypot Kippo"}

In the new, it hasn't:

      "analyzerType": "Network Honeyport Dionaea v0.1.0",
      "requestString": "", 

The analyzer Type "Network Honeyport Dionaea v0.1.0" is quite long to display in the table. If possible, shorten it in the backend or put it into the requestString field.

We decided to completely drop the python implementation of our honeypot data ingestion for our backend at DT and rewrite the code in Golang (thx @rverton). We'll follow up with a blog post when these changes go live in prod. We'll continue to use PEBA for our public API, mainly used for sicherheitstacho.eu, without ingestion capabilities though.

[
{
"id":1,
"date":"2016-12",
"attacksPerCountry":[
{
"country":"China",
"code":"CN",
"count":"80520605"
},
{
"country":"Viet Nam",
"code":"VN",
"count":"13787034"
},
{
"country":"United States",
"code":"US",
"count":"10273960"
},
{
"country":"Russia",
"code":"RU",
"count":"8590511"
},
{
"country":"Germany",
"code":"DE",
"count":"6254960"
},
{
"country":"Taiwan, Province of China",
"code":"TW",
"count":"3836209"
},
{
"country":"Egypt",
"code":"EG",
"count":"2853632"
},
{
"country":"Kazakhstan",
"code":"KZ",
"count":"2633969"
},
{
"country":"Korea, Republic of",
"code":"KR",
"count":"2103168"
},
{
"country":"France",
"code":"FR",
"count":"2059442"
},
{
"country":"Netherlands",
"code":"NL",
"count":"1855017"
},
{
"country":"Venezuela",
"code":"VE",
"count":"1649844"
},
{
"country":"Lithuania",
"code":"LT",
"count":"1622323"
},
{
"country":"Brazil",
"code":"BR",
"count":"1573446"
},
{
"country":"Ukraine",
"code":"UA",
"count":"1495057"
}
],
"attacksToTargetCountry":[
{
"country":"USA",
"code":"USA",
"targetCountries":[

        ]
     },
     {  
        "country":"ES",
        "code":"ES",
        "targetCountries":[  

        ]
     },
     {  
        "country":"PL",
        "code":"PL",
        "targetCountries":[  

        ]
     },
     {  
        "country":"HR",
        "code":"HR",
        "targetCountries":[  

        ]
     },
     {  
        "country":"RO",
        "code":"RO",
        "targetCountries":[  

        ]
     },
     {  
        "country":"GR",
        "code":"GR",
        "targetCountries":[  

        ]
     },
     {  
        "country":"ME",
        "code":"ME",
        "targetCountries":[  

        ]
     },
     {  
        "country":"US",
        "code":"US",
        "targetCountries":[  

        ]
     },
     {  
        "country":"CH",
        "code":"CH",
        "targetCountries":[  

        ]
     },
     {  
        "country":"MK",
        "code":"MK",
        "targetCountries":[  

        ]
     },
     {  
        "country":"HU",
        "code":"HU",
        "targetCountries":[  

        ]
     },
     {  
        "country":"AT",
        "code":"AT",
        "targetCountries":[  

        ]
     },
     {  
        "country":"VN",
        "code":"VN",
        "targetCountries":[  

        ]
     },
     {  
        "country":"FR",
        "code":"FR",
        "targetCountries":[  

        ]
     },
     {  
        "country":"SK",
        "code":"SK",
        "targetCountries":[  

        ]
     },
     {  
        "country":"IE",
        "code":"IE",
        "targetCountries":[  

        ]
     },
     {  
        "country":"GB",
        "code":"GB",
        "targetCountries":[  
           {  
              "country":"China",
              "code":"CN",
              "count":"10474"
           },
           {  
              "country":"United States",
              "code":"US",
              "count":"3242"
           },
           {  
              "country":"Chile",
              "code":"CL",
              "count":"2358"
           },
           {  
              "country":"Ukraine",
              "code":"UA",
              "count":"1369"
           },
           {  
              "country":"Netherlands",
              "code":"NL",
              "count":"937"
           },
           {  
              "country":"Germany",
              "code":"DE",
              "count":"785"
           },
           {  
              "country":"New Zealand",
              "code":"NZ",
              "count":"782"
           },
           {  
              "country":"Korea, Republic of",
              "code":"KR",
              "count":"628"
           },
           {  
              "country":"Canada",
              "code":"CA",
              "count":"598"
           },
           {  
              "country":"Viet Nam",
              "code":"VN",
              "count":"525"
           },
           {  
              "country":"United Kingdom",
              "code":"GB",
              "count":"252"
           },
           {  
              "country":"France",
              "code":"FR",
              "count":"219"
           },
           {  
              "country":"Russian Federation",
              "code":"RU",
              "count":"150"
           },
           {  
              "country":"Turkey",
              "code":"TR",
              "count":"113"
           },
           {  
              "country":"Cayman Islands",
              "code":"KY",
              "count":"101"
           }
        ]
     }
  ]

}
]

getPeerCountry(peerIdent):

Hi,

the current tacho makes three calls towards the new backend for displaying
the last alerts for the last minute, last hour and last 24 hours:

https://community.sicherheitstacho.eu:9443/alert/retrieveAlertsCount?time=1&out=json
https://community.sicherheitstacho.eu:9443/alert/retrieveAlertsCount?time=60&out=json
https://community.sicherheitstacho.eu:9443/alert/retrieveAlertsCount?time=1440&out=json

Is it possible to offer a special version for the tacho to reduce the unnecessary calls to a single one?

Something like:
https://community.sicherheitstacho.eu:9443/alert/retrieveAlertStats&out=json

Example response:

{
  "AlertsLastMinute": 39,
  "AlertsLastHour": 12313213,
  "AlertsLast24Hours": 13214145128496
}

As spoken today, please additionally support returning alert-docs from ES without a clientDomain-filter.

From client/sicherheittacho perspective, this is only needed for the endpoints:

/alert/topCountriesAttacks
/alert/retrieveAlertStats
/alert/retrieveAlertsCountWithType

optional (can be done on client side as there is anyway some steps to do regarding flagging the origin for stuff like coloring etc):

/alert/retrieveAlertsJson

Hi there,

In the the installation section of the MaxMind GeoIP the download.sh script no longer works.

download Maxmind's GeoIP database and copy it to the right destination

cd var/lib/GeoIP/
./download.sh
sudo cp *.dat /var/lib/GeoIP/

Maxmind has moved to GeoLite2.
https://dev.maxmind.com/geoip/geoip2/geolite2/

Regards,
Danny

[
{
"datasetAlertsPerMonth":{
"20170701":{
"ssh":200,
"web":500,
"esek":42
},
"20170702":{
"ssh":200,
"web":500,
"esek":42
},
"20170703":{
"ssh":200,
"web":500,
"esek":42
}
}
}
]