add name/email of logged in user to header of all pages

Add "find as you type" fields at the top of each column in /#!/user_export .

if a organization is deleted you still see memberships referencing the organisation. this in turn breaks the other stuff

example:
https:///api/1.0/organization_memberships
returns

{"city": "Membership City 318", "country": {"cc": "AT", "id": 10, "name": "Austria"}, "country_id": 10, "email": "[email protected]", "id": 318, "membership_role_id": 1, "mobile": "+43 2 318", "organization_id": 38, "phone": "+43 1 318", "sms_alerting": 0, "street": "Membership Street 318", "user_id": 183, "zip": "Membership ZIP 318"}

do_portal=# select * from organizations where id = 38;
-[ RECORD 1 ]---------+-----------------------------
created               | 2017-09-27 14:51:52.383778
updated               | 2017-10-24 08:08:18.119058
id                    | 38
organization_group_id | 
is_sla                | t
abbreviation          | Organization Abbreviation 38
old_ID                | 
full_name             | Organization Full Name 38
display_name          | Organization Display Name 38
mail_template         | EnglishReport
mail_times            | 3600
ts_deleted            | 2017-10-24 08:08:18.046181
deleted               | 1
parent_org_id         | 3

Check if https://github.com/Intevation/intelmq/issues/29 also applies

After the recent performance increases, a user can call
/#!/organizations/ and
/api/1.0/organizations/
if the user is not allowed to do so. Previously to the changes the requests were correctly denied with 403.

For new accounts an email with the password is sent. Instead, the usual token-based approach should be used.

User gets an email with a link. The page offers to set an initial password.

After software or database changes the update process has to work.
Also the process of how to update has to be documented.

Editing an organizations' details gives the shown message.

For example the columns created, updated, ts_delete of table organizations all have the type timestamp without time zone -> we need to check if this is sufficient or we need to change it

When the session timeouts (or cookies are deleted) and the user tries to load the user list page /#!/user_list 4 "unauthorized" banners are shown to the user. The number of banners is the number of failed requests (for the organization list it's only one)

The banners even hide the login form.

Proposal:

Instead of showing the message multiple times, only show it once
Use a more helpful error message, eg. "Session timeout. Please login again."

From the logs:

WARNING in decorators [/home/doportal/do-portal/app/api/decorators.py:26]:
Using the json_response decorator is deprecated.Please use app.core.ApiResponse.

If a new organisation is created it of course hasnt any members. But therefore the field "Memberships" is not displayed making it not possible to add contacts from the Cannot add contacts to newly created organisation view e.g. /#!/organizations/5072 .
Even if there is no contact/membership in an organisation the field "Memberships" should be displayed nevertheless.

do-portal | ERROR in mixins [/home/cert/do-portal/app/utils/mixins.py:39]:
do-portal | 'NoneType' object has no attribute 'abbreviation'

think that came up the first time when we started displaying parent_org_name

The email templates in app/templates/auth/email/ do have multiple issues:

• the signature is hardcoded and not configurable. Currently one template has AEC, the others have CERT-EU signatures
• the signature separator misses a blank at the end
• some templates use Dear {name}, others use Dear {email}, all should use the name.

Setting an email address with upper case characters for any user pressing save for the user gives "Attribute error. Invalid email, phone or mobile?"

But users don't have phone or mobile numbers, only the email can be invalid.

The actual bug will be tracked in another issue.

Requesting contacts (#!/user_list) needs 327 queries and 7 seconds to load. This is too many/ too long.

Scenario: A non-admin user calls https://cp-aec-stg.cert.at/#!/organizations/<id> for it's own organization. There is a parent organization and the user has no right to view it.

As the page requests information about the parent organization to display the "Parent Organization Abbreviation", this causes a 403 error.

Proposal @certrik we could allow the user to view only the name for the parent organization, so the user does not see an error, and only the needed information.

In the organization view the parent organization's name should be displayed.

see #11

There are variable error 502 when accessing /#!/organization_list or /#!/user_list. See screenshot ...

error not shown on creating new contact

if a new contact/user with invalid data is created and the server returns 422
{"message": "Attribute error. Invalid email, phone or mobile?password has to contain an upper case letter"}

the frontend ignores the error an does nothing

E.g. The following org hierarchy:

Org 29
    Org 35
        Org 36

OrgAdmin of Org 29 should be able set the parent Org of Org 36 to any org he can see (i.e. 29, 35).

To clarify ... this is only possible at the creation time of the organization at the moment. But this should also be possible after the initial creation in the "edit" menu of the organization.

It is possible to submit "empty" ripe handles

Roles that are deleted still are viewed in the contact details.
If the role is added again, it is shown twice (trice) in the contact details and cannot be deleted (Error: Resource not found) or edited (Error: <html><head><title>502 Bad Gateway</title></head> <body bgcolor="white"><center><h1>502 Bad Gateway</h1></center> <hr><center>nginx/1.6.2</center></body></html> ). The original role can be deleted but is still shown in the contact details but not in the organizational details (but only after logging out and in again).

both repos (do+ cp) now have a topic-docker branch.

https://github.com/certat/do-portal/blob/topic-docker/docs/docker.md

for a non-docker setup follow the instructions in the respective Dockerfiles

https://github.com/certat/do-portal/blob/topic-docker/Dockerfile
https://github.com/certat/customer-portal/blob/topic-docker/Dockerfile

Wenn eine 502 Bad Gateway zum Backend auftritt (zB Backend ist beendet), wird im Frontend nur ein roter Balken angezeigt, ohne Fehlermeldung

do-portal

laeuft auf port 5001

customer-portal

laeuft auf port 5002

/* ./config/envs/devel.json */
{
  "version": "0.7.0",
  "webServiceUrl": "http://127.0.0.1:5001/cp/1.0",
  "authUrl": "http://127.0.0.1:5001/auth"
}

wenn man nun das login form submitten will

OPTIONS http://127.0.0.1:5001/auth/login

response headers

HTTP/1.0 200 OK
Content-Type: text/html; charset=utf-8
Allow: POST, OPTIONS
Access-Control-Allow-Origin: http://127.0.0.1:5002
access-control-allow-methods: POST, OPTIONS
access-control-max-age: 21600
access-control-allow-credentials: true
access-control-allow-headers: Content-Type, Accept, Authorization, Origin, CP-TOTP-Required
Access-Control-Expose-Headers: Content-Type, Accept, Authorization, Origin, CP-TOTP-Required
Content-Length: 0
Server: Werkzeug/0.11.15 Python/3.5.3
Date: Mon, 13 Aug 2018 09:58:36 GMT

app zeigt nur ajax error als notification an weil das exception object so aussieht:
Object { data: null, status: -1, headers: headersGetter/<(), config: Object, statusText: "", xhrStatus: "error" }

das kommt von hier:

/* app/scripts/services/auth.js */
      login: function (credentials) {
        return $http.post(config.apiConfig.authUrl + '/login', credentials)
          .then(cacheSession);
      },

prinzipiell ist das eine fehlkonfiguration zwischen do-portal und customer-portal die nicht ganz trivial zu finden ist.

As role "Adminstrator Organisation" of an organization the error: "You don't have the permissions to access the requested resource. It is either read-protected or not readable by the server." is thrown. It is still rightfully possible to edited the organizational contacts and structures.
I assume that the error is thrown because the query somehow tries to access a resource from the parent of this organization which is not granted.

We do not need some components such as malware analysis, mailman, cerlery etc. The requirements should be optional and the interfaces not exposed, the components not displayed in the UI.

With a current version of flask-login:

AttributeError: 'LoginManager' object has no attribute 'token_loader'

See https://github.com/maxcountryman/flask-login/blob/594e5d77c1101a103c76caf46895cea350e14287/CHANGES#L13-L16

It seems that for users only lower case email addresses are allowed, which is not intuitive as email addresses are (according to RFC) case sensitive and in practice used case-insensitive. For memberships, upper case characters are allowed.

So I propose to allow upper case characters for users' email addresses too.

Do not allow removal of an organisation if the orga has child orgas.

Obtaining mailmanclient from git+https://gitlab.com/mailman/mailmanclient#egg=mailmanclient (from -r requirements.txt (line 44))
  Cloning https://gitlab.com/mailman/mailmanclient to ./src/mailmanclient
    Complete output from command python setup.py egg_info:
    Python 3.5.0 or better is required�
    88% |�����������������������������
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /home/cp-server/do-portal/src/mailmanclient/

We are using Python 3.4

This is a release blocker.

On https://cp-aec.cert.at/#!/users/6 the CoC should not be loaded by default, only when the user actually accesses it.

The corresponding API call is /api/1.0/users/<id>/memberships AFAIK.

@certrik I guess it also applies to the S/MIME certificate?

Betreff ist im Moment "reset password".
Soll lauten "Austrian Energy CERT - Kontaktdatenbank:
Account-Aktivierung/Passwort-Reset"

Text soll Deutsch und Englisch sein:
"
Lieber ,

nur mehr ein weiterer Schritt um dein initiales Passwort festzulegen
oder dein altes neu zu setzen.
Klicke auf folgenden Link:
https:///auth/activate-account/eyJleHAiOjE1Mzk2MDE3NjksImFsZyI6IkhTMjU2IiwiaWF0IjoxNTM5NjAwODY5fQ.eyJ1c2VyX2lkIjo0MjJ9.Zb9ruNfDCItYEwr0qHoOxtxzSX0BNLgq5qBhJvaOvYU

Dear ,

only one more step to set your initial password or reset your old one.
Click on following link:
https:///auth/activate-account/eyJleHAiOjE1Mzk2MDE3NjksImFsZyI6IkhTMjU2IiwiaWF0IjoxNTM5NjAwODY5fQ.eyJ1c2VyX2lkIjo0MjJ9.Zb9ruNfDCItYEwr0qHoOxtxzSX0BNLgq5qBhJvaOvYU

--
// Austrian Energy CERT - https://www.energy-cert.at/
// T: +43 1 505 6416 92
// E: team[at]energy-cert.at (edited for anti-spam issues [at] should be @ of course)
// Eine Initiative der österreichischen
// Elektrizitäts- und Erdgaswirtschaft und
// nic.at GmbH, Firmenbuchnummer 172568b, LG Salzburg

From /var/log/uwsgi/app/doportal.log

ERROR in mixins [./app/utils/mixins.py:39]:
'NoneType' object has no attribute 'abbreviation'

As it works anyway I'd not consider it as release blocker

We get a JS error because of an undefined property:

TypeError: a.organizations[d.organization_id] is undefined
Stack trace:
@https://cp-aec.cert.at/scripts/scripts.96600f9a.js:1:16654
@https://cp-aec.cert.at/scripts/scripts.96600f9a.js:1:16592
@https://cp-aec.cert.at/scripts/scripts.96600f9a.js:1:16570
i@https://cp-aec.cert.at/scripts/vendor.d5ec3447.js:5:31837
k/<@https://cp-aec.cert.at/scripts/vendor.d5ec3447.js:6:296
$digest@https://cp-aec.cert.at/scripts/vendor.d5ec3447.js:6:6128
$apply@https://cp-aec.cert.at/scripts/vendor.d5ec3447.js:6:8120
g@https://cp-aec.cert.at/scripts/vendor.d5ec3447.js:5:14309
r@https://cp-aec.cert.at/scripts/vendor.d5ec3447.js:5:16872
Yb/</w.onload@https://cp-aec.cert.at/scripts/vendor.d5ec3447.js:5:17296
 Possibly unhandled rejection: {}
vendor.d5ec3447.js:5:25613
e/<
https://cp-aec.cert.at/scripts/vendor.d5ec3447.js:5:25613
Lb/this.$get</<
https://cp-aec.cert.at/scripts/vendor.d5ec3447.js:5:10222
j
https://cp-aec.cert.at/scripts/vendor.d5ec3447.js:6:116
$digest
https://cp-aec.cert.at/scripts/vendor.d5ec3447.js:6:6128
$apply
https://cp-aec.cert.at/scripts/vendor.d5ec3447.js:6:8120
g
https://cp-aec.cert.at/scripts/vendor.d5ec3447.js:5:14309
r
https://cp-aec.cert.at/scripts/vendor.d5ec3447.js:5:16872
Yb/</w.onload
https://cp-aec.cert.at/scripts/vendor.d5ec3447.js:5:17296

Not sure what the reason is. The user_list has successfully been shown at least once after the upgrade.

If the data is bad it shouldn't have been accepted.

1. call https://cp-aec-stg.cert.at/#!/users/15
2. change the email address to e.g. [email protected]
3. click save
4. see green User saved message
5. reload page
6. still old email address there

Previously all users got a role_id (by default 3 == constituent). Now the default value is NULL for role_id.

-> Does this cause (permissions) errors/problems
Why is this so?
Why can the role be NULL at all? Shouldn't it be NOT NULL in the database?

pls implement this test cases (which can be run at every import of the RIPE DB):

• import a faulty { inetnum,role,organisation} RIPE file (with syntax errors) --> the parser should detect this and stack trace / report an error in parsing
• import a working {inetnum,role,organisation} file and for sample N (default 100) inetnum ranges. Query the abuse_c via stats.ripe.net and compare with what the DB would give us now. There should be 0 difference.
• put the import into a cron job on a VM and have it fetch & import the RIPE files daily and send stack traces / error reports via email so that we are exposed to parsing issues.

We need a simple first RESTful API endpoint to query IPs or netblocks for abuse contacts.
Please

• create this RESTful API endpoint
• document it
• and cross check with @wagner-certat so that he can use it

Test for this is that it works on our intelmq-dev instance and a intelmq bot exists which can query the DB.

Link RIPE-orgas to do-orgas via RIPE-handle following the principle described in https://github.com/certat/do-portal/wiki/fody-implementation#data-structure .

When changing organization details (Name, Abbreviation) I get a 502 Bad Gateway.

Add active full-text search with search field for name, organization, city, email address.

Currently every dependency in requirements.txt has a hard version requirement.

We need to relax this in order to get bugfixes the needed libraries.

The email address of the currently logged in user is shown in the header Logout (...). The address disappears after reloading the page.

If editing a user, for example /#!/users/14, it would be good to be able to jump back to the belonging organization. Not sure about the best way to do it yet. But would improve useability.

The data gained from #37 should be available in the user interface.
Example: