decred / politeia Goto Github PK
View Code? Open in Web Editor NEWLicense: ISC License
License: ISC License
Currently the proposal title is not part of the merkle and therefore signature, so it's possible for a user to submit a perfectly valid proposal to Politeia with a bad/offensive title, have it censored, and be able to make a case to others that the valid proposal was unfairly censored.
To address this, the title will be embedded in the index.md
file as the first line with text by the client, and www should extract it, strip any markdown characters (if present), and then set it on the ProposalRecord before sending to politeiad.
politeiad/backend/gitbe/git.go:108:6:warning: stderrError declared but not used (unused)
politeiad/backend/gitbe/git.go:89:6:warning: stdoutError declared but not used (unused)
Politeiad rejects proposal names that are greater than 80 characters. There should be a policy in www for this. www is also validating what characters are allowed, so we should have a separate policy that includes the regex of supported characters as well.
All files required need to be read into a global variable on startup
See https://golang.org/pkg/html/template/#Must
Move all templates to templates/ directory.
Use https://godoc.org/net/http/httptest to test routes which include the logic within www.go
instead of the current backend_* tests.
We need to add a JSON mode to the verification tool. The idea is that we should be able to spit out the JSON once a proposal is submitted and then feed that directly into the tool for verification.
We probably need something like -jsonin to import the JSON submission reply. When that flag is set -k -m etc should cause the tool to abort. We should only do one or the other.
We also need a -jsonout flag that replies in some dort of JSON format. We can then use this tool as a backend to some GUI.
$ politeia -v -testnet -h 127.0.0.1 getunvetted 959c855726c75270ef435c67c5f7fa86458084f3dde62e4e9462b46b15ec2147
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x6e85d7]
goroutine 1 [running]:
github.com/decred/politeia/api/v1.Verify(0xc04243e340, 0xb, 0xc04243e350, 0x9, 0x34d3653afe46033, 0x455d02561df587f, 0x7f53a817b75154ac, 0xc95b2b1797abd27b, 0x7f0054865f7ae017, 0xf0acaf7ba5fe8cc2, ...)
C:/projects/gopath/src/github.com/decred/politeia/api/v1/v1.go:96 +0x407
main.getUnvetted(0x0, 0x0)
C:/projects/gopath/src/github.com/decred/politeia/cmd/politeia/politeia.go:487 +0x9e1
main._main(0xc04202bf70, 0x4044ab)
C:/projects/gopath/src/github.com/decred/politeia/cmd/politeia/politeia.go:782 +0x3ca
main.main()
C:/projects/gopath/src/github.com/decred/politeia/cmd/politeia/politeia.go:798 +0x2d
Use (make sure to glide get it): https://godoc.org/github.com/dajohi/goemail
#72 kills me on the inside and we need to move this to a file. This must not be in a relative path to the binary. I suggest we put it in the same directory as the config file and look for it, by default, there.
Politeiad is returning 400 Bad Request for errors due to user input, but the error message is a string rather than a numeric error code, which would be easier for www to process. Errors should be reworked to return JSON with 400 responses with an error code and, optionally, a human-readable error message.
We have had some vague ideas on how to do payments when submitting a proposal. Let's debate and nail down the method in this issue.
Currently there is no way for the client to know which user created which proposal. We need www to support user-proposal associations in the database, and to send the user id (and other identifying information) along with each proposal record.
The politeiad Inventory command is very expensive time and bandwidth wise. We, therefore, need to cache proposal records at start-of-day. At this time we should shoot for an implementation that fires off the Inventory command and then places the vetted and unvetted branches in memory use a map with the censorship token as the key). Currently, the Inventory call does not filter and will always reply with everything it has. That is a separate issue that will be addressed once we get the code to work.
Let's not cache to disk yet because reconcile code is always hard.
browser was stuck in what seems like a loop.. hitting server every ~200ms
2017-10-19 15:07:02.110 [INF] PWWW: 70.114.160.212 via 10.135.7.135:48530 GET / HTTP/1.1
2017-10-19 15:07:02.301 [TRC] PWWW: GET / HTTP/1.1
Host: test-proposals.decred.org
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.8
Connection: upgrade
Cookie: _gorilla_csrf=MTUwNzIyMDE0OXxJa3BLUkRVME5rZEtLM0pJUkRONFNHSkpiWEYxV2xSQlpsUndRMnBJZUZGeFZWaFZPRWQzVHpOMFVVRTlJZ289fK8mI9mx3K8Wbzywd3HwAtar3dRKDyMwsQ7O_I7TlwPl
Dnt: 1
Referer: https://test-proposals.decred.org/user/signup
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36
X-Forwarded-For: 70.114.160.212
2017-10-19 15:07:02.301 [INF] PWWW: 70.114.160.212 via 10.135.7.135:48532 GET / HTTP/1.1
2017-10-19 15:07:02.473 [TRC] PWWW: GET / HTTP/1.1
Host: test-proposals.decred.org
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.8
Connection: upgrade
Cookie: _gorilla_csrf=MTUwNzIyMDE0OXxJa3BLUkRVME5rZEtLM0pJUkRONFNHSkpiWEYxV2xSQlpsUndRMnBJZUZGeFZWaFZPRWQzVHpOMFVVRTlJZ289fK8mI9mx3K8Wbzywd3HwAtar3dRKDyMwsQ7O_I7TlwPl
Dnt: 1
Referer: https://test-proposals.decred.org/user/signup
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36
X-Forwarded-For: 70.114.160.212
2017-10-19 15:07:02.475 [INF] PWWW: 70.114.160.212 via 10.135.7.135:48534 GET / HTTP/1.1
2017-10-19 15:07:02.654 [TRC] PWWW: GET / HTTP/1.1
Host: test-proposals.decred.org
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.8
Connection: upgrade
Cookie: _gorilla_csrf=MTUwNzIyMDE0OXxJa3BLUkRVME5rZEtLM0pJUkRONFNHSkpiWEYxV2xSQlpsUndRMnBJZUZGeFZWaFZPRWQzVHpOMFVVRTlJZ289fK8mI9mx3K8Wbzywd3HwAtar3dRKDyMwsQ7O_I7TlwPl
Dnt: 1
Referer: https://test-proposals.decred.org/user/signup
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36
X-Forwarded-For: 70.114.160.212
2017-10-19 15:07:02.654 [INF] PWWW: 70.114.160.212 via 10.135.7.135:48536 GET / HTTP/1.1
2017-10-19 15:07:02.825 [TRC] PWWW: GET / HTTP/1.1
Host: test-proposals.decred.org
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.8
Connection: upgrade
Cookie: _gorilla_csrf=MTUwNzIyMDE0OXxJa3BLUkRVME5rZEtLM0pJUkRONFNHSkpiWEYxV2xSQlpsUndRMnBJZUZGeFZWaFZPRWQzVHpOMFVVRTlJZ289fK8mI9mx3K8Wbzywd3HwAtar3dRKDyMwsQ7O_I7TlwPl
Dnt: 1
Referer: https://test-proposals.decred.org/user/signup
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36
X-Forwarded-For: 70.114.160.212
2017-10-19 15:07:02.826 [INF] PWWW: 70.114.160.212 via 10.135.7.135:48538 GET / HTTP/1.1
There is no way to verify that the signature provided as part of the censorship token data is signed by Politeia.
If a user says his proposal has been unjustly censored by Politeia, it would be nice to have a CLI or some other tool for others to easily verify his claim.
I think this is an nginx issue; when I open https://test-proposals.decred.org and navigate to a proposal, it correctly opens. But if I go to the address bar and press enter to navigate directly to the same proposal, it sometimes returns 200 OK
for the request but there's no response data.
This is where we are submitting the proposal. We probably need to show a text box where the user can enter md and upload some files and do a "view" to see it rendered. I am open to other ideas.
I think GitHub has nailed the md editing quite well so let's borrow some ideas from them. Maybe there is even some code out there.
Add a route that adds a new unvetted proposal to the repo. This is also the right location to start enforcing some policy. We need a max file size, max files and files types (make these config settings with sane defaults).
Currently, politeiad allows only text and png. We should add svg to that mix.
Don't forget to add the new proposal to the www cache!
/proposals/{token}/setstatus
needs to be fixed to instead be /proposals/{token}/setstatus/{proposalstatus}
to have consistent parameters.
ProcessNewUser
into UpdateUserVerification
and InsertNewUser
within backend.go
webserveraddress
to webhost
Add a RPC that returns policy settings of www. Use GET route /api/v1/policy and reply with a JSON message like so:
type PolicyReply struct {
MaxImages uint
MaxImageSize uint
MaxMD uint
MaxMDSize uint
ValidMIMETypes []string
}
We should have a new policy representing the comment size limit, either in bytes or characters that is sent in /v1/policy
response.
If mailhost, mailuser, and mailpass are populated, webserveraddress should be populated too.
When writing the router I added a file browser and it points to the binary location. Make this point to an actual directory.
2 issues:
I hate capthca as much as the next guy but we need to prevent robots and floods.
An idea was thrown out there to require a user to pay a nominal fee to sign up. I kind of like that idea as well.
Thoughts?
If you create a JPG/BMP and save as a PNG and upload it with a proposal, the server will return a 500 error. It should return a valid invalid-mime-type error code along with the detected mime type of the file.
Add the default/user/admin proposal lists from the mockup. The unreviewed admin list should have the ability to Publish/Censor.
in at least politeiad.conf
Currently the /
route creates a new session and returns route info, but /v1/user/me
is used by the client to check if a session already exists. So the order the client must make the calls is:
/v1/user/me
to check if user has session/
to get the route info ({ route: "/v1" }
) and create a sessionThat ordering doesn't make sense, because the client has to know which version of the API to use before getting that information from the API. We need a separate endpoint to create a new session.
UserError
error type in backend.go
(see https://github.com/decred/politeia/blob/master/politeiad/backend/gitbe/git.go#L17-L28 for reference) and use that to represent all user errors.www.go
to check the error type returned by functions in backend.go
and decide on the appropriate action. Internal (real) errors should be logged with the originating IP address. User errors should be returned as part of the JSON. Both errors should return an appropriate HTTP status error code.Pagination should be done like reddit's API, with support for the following:
after
: the id of the last proposal on the previous pagebefore
: used for navigating backwards through the list in the UI, it's the id of the first proposal on the previous pagelimit
: (optional) the number of proposals to fetch in the pagelogging.go
currently uses http.Request.RemoteAddr
, but that apparently won't work behind nginx, so fetch it using the X-Forwarded-For header. Example: https://stackoverflow.com/a/26223739
The front page is a list of getvetted input. We should probably make the proposals clickable so that we can drill down on them. Ultimately we need to render the md with graphics in a reasonable manner.
We sometimes end up with the browser sending multiple cookie values.
It's not yet clear if this is a client or server issue.
Currently, we stuff everything in a local leveldb. It is an interface and we should add a redis option in order to be able to horizontally scale and be more container-y.
Administrative actions (like publishing/censoring a proposal) should probably not be absolutely available to a single admin level user, because it opens up potential for abuse. Similar to Stack Overflow moderation, it's probably better if an administrative action (especially a non-reversible action) can only be performed after a few different users have voted to take it.
It might be beneficial in the long run to eventually adopt a few other ideas from Stack Overflow's moderation system, like different levels of moderator permissions and elections for new moderators, to utilize the community to make Politeia more self-sustaining.
@davecgh mentioned this has been discussed, so I'm just creating an issue to capture it.
Currently there is no support for the admin providing a reason to the user for censoring a proposal.
When a regular user accesses proposal details for unreviewed/censored proposals, the server does not send the markdown or images. It should also not send the title.
Please add a node.js GUI that transforms politeiawww JSON to pretty pretty UX/UI.
www shouldn't required politeiad to be running when it starts up, so inventory shouldn't be loaded on startup but on the first proposal-related request.
In order to render the main page we need the getvetted route. Please do something similar as getunvetted. We may want to create a separate map for unvetted proposals.
Currently unreviewed and censored proposals are accessible via direct link to the public, and this could be a liability if a malicious user uploads illegal images. Unvetted proposals should only be accessible to admin users and direct links to unvetted proposals (when accessed by non-admins) should the censorship token & status. We could return 404 for those proposals, but if we want to implement something like #97, then that information could be shared in a publicly accessible way through the proposal details page.
Refactor www.go
and backend.go
so that functions in backend.go
are smaller and more cohesive and www.go
handles the incoming commands and creates the outgoing replies.
We need to be able to paginate api endpoints for proposal lists.
I recommend the approach reddit uses.
A list is sorted in some way (let's assume new)
To get a page, I specify the limit (max number of items returned) and an optional start/end point.
You could have a default limit of 20.
For these examples, assume the list is sorted by newest first.
To get a page of i items I fetch
/vetted?limit=20
To get the next oldest page, I would fetch:
/vetted?limit=20&before=<idofthelastiteminpreviouspage>
A meta key can be added to the response as a sibling to proposals
{
proposals: [...]
meta: { total: 9123 }
}
Before the real go-live, we need to come up with a standard format (or at least a minimum set of information) that is expected of all proposals.
You can't access the /proposals/{token}
url directly by copying/pasting into the browser or clicking a link, because the route doesn't correctly support url params.
Currently we're only performing strict text comparison of emails, so you can hit /v1/user/new
to spam one email address by using different text variations which will deliver to the same email address. For example:
[email protected]
[email protected]
[email protected]
[email protected]
...
Merkle verification can fail for newly submitted proposals when running on Windows with the following git config:
git config --global core.autocrlf true
Steps to reproduce:
\r\n
characters.Politeiad should set core.autocrlf
to false
locally within both vetted and unvetted repos to avoid this issue where git automatically converts between \r\n
and \n
.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.