ptt / pttbbs Goto Github PK

https://www.ptt.cc/index.source.html 中

若您有需要自己架設一個 Ptt BBS ，您可以參考 INSTALL 這份文件；相關的程式碼以及所需要的附件，
您可以透過 Subversion Repository 直接取得 (位於 http://opensvn.csie.org/pttbbs) ，或是透過 
ftp://ptt2.cc 取得每日自動做成的 tar ball 。 

連結均過時。

建議使用 2.6.x 的核心

亦過時等等…

This is part of the issue #5

To my understanding of the code:
(in edit_post in bbs.c)

Currently comments (recommend / boo / comment) is directly attached to the end of file.
Currently the method to do editing main-content is loading to temporal mem, and then
use append_merge_replace to sync back to the file.

I would like to propose the following new policy to do the rewriting:

1. For each content, there are 3 types of operators:
   edit, comment, comment-reply-from-poster.

2. "edit" can do editing only on main-content

3. comments is the same as current settings (recommend / boo / comment),
   but each comment is stored in a separate record.

4. comment-reply-from-poster let the poster be able to choose the comment-to-reply, stored as separated record, but with information pointing to the comment-to-reply.

5. Given a user retrieving the content at a given time:
   GOAL: retrieve consistent content partially based on screen size.
   The order of the content: main-content, comments and the corresponding comment-replies.

   For the editable content (main-content, comment-replies), we also record the mtime to check whether the content is updated before scrolling screen.

For each old-content:

1. separate as main-content and comments based on the separator
   "文章網址 / comments"

2. for all the main-content after comments: treat as comment-replies.

TODO:

1. random sampling some articles from BBSMovie and see whether there is conflict about the settings.
   (Or we may have already known some bad-format articles and we can see how to deal with those exceptions～)

2. get the list of contents not following single "文章網址 / comments" setup
   and check further how do we deal with those contents.

郵箱可訪問，或者請幫忙註銷 [email protected] 重新註冊

Some ideas to modernize the code:

• Use arc4random and co.
• Re-align structures and drop the packed attribute
• Use crypt_r w/ SHA256 or SHA512
• IPv6 support?

If the user crossposts (^X) more than three times,
one hour later his articles will be deleted,
and he will be charged $1000,
which he is unable to pay.
形同毀帳。

倒不如於第四 crosspost, simply not allow him to post the fourth
crosspost. 比毀掉人家帳號好多了。

且使用者每次 crosspost 都看到同樣警語。
惟某一天超越同文三次門檻即毀帳，
非常不應該的。

Currently there are two systems in place to prevent abuse:

1. Opt-in per-board limitations on push/recommend intervals
2. System-wide "cooldown" mechanic for board opt-in and mandatory limitations on accounts with rejected articles.

The timer for the first system is non-persistent, meaning it is reset when the user logs off.
The timer for the second system is persistent in shared memory, but the rules for it are slightly opaque, and it only has 15 second resolution.

It may be a good trade-off to use the persistent timer of the second system for the first one as well. If the resolution is unacceptable, then move out the rejected-article counter to restore it to per-second resolution.

https://github.com/ptt/pttbbs/blob/master/common/sys/lock.c#L15

Found several lines using PttLock while already did lseek.
(but some didnt use lseek before PttLock)

Not exactly sure whether offset of lseek is the same as the fnctl,
and not exactly sure whether fcntl F_SETLKW affects seek-pointer though.

The following lines use PttLock F_WRLCK

https://github.com/ptt/pttbbs/blob/master/common/sys/record.c#L84 (after lseek)
https://github.com/ptt/pttbbs/blob/master/common/sys/record.c#L133 (after lseek)
https://github.com/ptt/pttbbs/blob/master/common/sys/record.c#L195 (no lseek)
https://github.com/ptt/pttbbs/blob/master/common/sys/record.c#L261 (no lseek)

The following lines use PttLock F_UNLCK
https://github.com/ptt/pttbbs/blob/master/common/sys/record.c#L86 (after lseek)
https://github.com/ptt/pttbbs/blob/master/common/sys/record.c#L149 (after lseek)
https://github.com/ptt/pttbbs/blob/master/common/sys/record.c#L223 (after lseek)
https://github.com/ptt/pttbbs/blob/master/common/sys/record.c#L275 (after lseek)

I would like to point out that identifiers like “_LIBBBS_H_” and “_STATISTIC_H_” do not fit to the expected naming convention of the C++ language standard.
Would you like to adjust your selection for unique names?

目前只允許下列站點使用 WebSocket，

然而 https://robertabcd.github.io 已經很久沒有更新了，

是否可以開放其他客戶端連線？

The plan is to do account validation with cell phone verification via SMS or AOTP solutions.
The solutions will almost all be using HTTP. Cell phone verification should be a separate level.

First, we need data fields to store the data. I suggest using the _unused1 field in struct userec_t for extra permissions. We might want to combine it with the existing userlevel into a 64bit unsigned value. For the phone number, the old _unused_phone field is still there.

Second, we'll need to duplicate the existing email validation system, but with separate storage for in-flight validation codes, as well as the central database used to check for duplicates.

Third, integration with service provider. Probably use libcurl, either in-process, or queued request in a separate daemon.

More to come later....

Rationale

希望能直接在 PTT 上支援目前實作於各 PTT 行動客戶端應用的黑名單功能，讓使用者能夠在瀏覽文章時知道哪些 ID 在黑名單內

這個算是很常用(?)的功能應該要直接實作在站台上

實作細節

• 吻合的 ID 顏色調暗
• 吻合 ID 發表的推文文字顏色調暗
• 在文章瀏覽畫面中有快捷鍵（例：Ctrl+B）可以輸入要加進黑名單中的 ID

Extra

• 像是 Adblock 規則一樣，允許分享自己組建的，針對特定類型使用者的黑名單供他人訂閱使用

Currently there are at least two code paths that can delete users:

1. When a new user registers, the register system will look for an empty slot, and if one cannot be found, it will proceed to "expire" an old user.
2. The reaper tool under util, which "expires" users wholesale.

The user deletion code is written into these code paths, without any sharing. Over time, as the reaper tool didn't get much usage, the differences grew.

The idea is to move common code for user deletion under common/bbs.

Is there anyone rewriting the code base of ptt?
If no one is doing that, I am going to give it a try.
The only reason I want to rewrite is quite boring. Love for ptt.
And the code base is quite old actually...
The goal is to avoid any change to user experience meanwhile improve the architecture of ptt to support some feature like API, and, writing in a more modern language.
Golang is my preference, so I would like to write it in Golang.
However, I don't think it is helpful to Ptt as the system is pretty mature and large. Too large that any downtime of it is not acceptable.
But I do think the sustainable development is necessary to Ptt too.
What do you think?

PS: To issue #4, I'm so sorry for my speech. Not meaning to attack Ptt, just a discussion of the possible vulnerability.

貌似我这里GBK有一定的问题...

With SO_REUSEPORT, we can do socket sharding, i.e. have one accept queue per socket/process, instead of sharing one among all workers.

See https://www.nginx.com/blog/socket-sharding-nginx-release-1-9-1/

假如你是用 GCP 或者是某些雲端服務的話，可以先測試

telnet gmail-smtp-in.l.google.com. 25 (或是 dig gmail.com 找個喜歡的 mail server 測試)

假如沒有看到像是 220 xxxxxx 之類的訊息的話，那代表這個提供商預設 Ban 掉 Port 25。

看起來應該是在 Postfix 裡面把預設連線到 25 的設定改掉就行了，不過我找了一下沒找到設定，所以我的作法是把 GCP 改掉。

As seen here: https://github.com/ptt/pttbbs/blob/master/daemon/regmaild/regmaild.c#L218
when the user ID is changed, it's regemaildb entry is not updated.

在看板中輸入文章 index 時，如果這個 index 跟目前畫面是在同一頁的話，最下面「文章選讀」的 bar 會消失，參考下面兩張圖：

1. 剛進入看板，下面的 bar 還在
   

2. 輸入 414[enter] 後，下面的 bar 消失了
   

不確定這是不是 PTT 本來就預期的效果，還是是 bug。就在這裡回報給各位參考了。

另外，消失的 bar 可能會讓 PTT 爬蟲不好判定有沒有下載完整個頁面。不過這點可能不是 PTT 需要考量的因素，各位也可以忽略這點。

In some places, such as email verification, the length is hardcoded, instead of based on the field in struct userec_t.

Need to clean this up.

這個其實不是bug, 也不是ptt問題. 只是突然想到的, 但是不知道去那建議, 就當異想天開好了. 看到ptt網頁版有人提到太多跳板IP張貼特定文章, 尤其某些立場很明顯的Id. 想到的方法是或許ptt可以考慮實作類似Rate Limit或IP Throttling的機制. 條件是跳版範圍越大, 跳板次數太多, 延長其張貼文章的間隔時間. 例如說, 某位使用者IP的國家登入範圍包括

英國, 法國, 德國, 美國, ..., 日本

張貼文章次數的頻率超過某個treshold, 或是採兩者混合統計(或類似可以加入當作機制設定的因素).

則該使用者每次張貼文章的時間間隔呈採Exponential backoff計算.

The idea is to validate and bind one account to one 自然人憑證.

It would be something like having our system send a one time token to the user,
who then signs it with their certificate, and sends it back, along with identifying data
of the certificate (common name + serial number).

The 自然人憑證 should be fetched via MOICA LDAP service ( ldap://moica.nat.gov.tw ),
and validated against MOICA certificate chain and CRL/OCSP.

Additionally, X509v3 Subject Directory Attributes should be validated as follows:

1. Should have object 2.16.886.1.100.2.1 (OID id-chtpki-at-subjectType), and
2. Value of this object should be set to 2.16.886.1.100.3.1.1 (OID id-chtpki-et-citizen) (國民) or 2.16.886.1.100.3.1.9) (OID id-chtpki-et-alienResident) (外國人)

Once validation passes, a unique identifier of the certificate / person should be kept (common name + type (citizen or alienResident) + 2.16.886.1.100.2.51 (身分證末4碼)).

On the server, we might want to do this as a separate microservice, as we might not want to pull in OpenSSL and LDAP dependencies, and also because of the network access.

make BBSHOME=/home/bbs TARGET_OSNAME=Linux all install
===> common
===> common/bbs
bmake[2]: exec(building) failed (No such file or directory)
*** Error code 1

Stop.
bmake[2]: stopped in /home/bbs/pttbbs/common/bbs
*** Error code 1

Stop.
bmake[1]: stopped in /home/bbs/pttbbs/common
*** Error code 1

Stop.
bmake: stopped in /home/bbs/pttbbs

使用者一旦誤按
●(F)orward 設定信箱自動轉寄
就無法離去。連 CTRL-C 會摧毀原設定。

Would you like to add more error handling for return values from functions like the following?

• malloc ⇒ buildchilds
• printf ⇒ check
• strdup ⇒ add_hash

有使用者對 2014 年底後實施的登入次數累計方法有以下疑問:
https://www.ptt.cc/bbs/PttBug/M.1622376938.A.CC0.html

故把問題轉錄過來，也順便請教看看其他開發者對此有什麼看法?

我访问了好一会你们的这个网站,发现如果我想留言或者想和你们做交流的话是没法做交流的,请问你们需要做一个可登录的功能吗

同標題，
OS： macOS 10.15.5(19F101)
瀏覽器：Chrome 83.0.4103.116

pttbbs/common/sys/utf8.c裡面的utf2ucs沒辦法支援4 byte長度的utf8
難道不支援的原因只是因為要配合用big5編碼的使用者？

How to reproduce
use this https://github.com/kcwu/pttbbs-fuzzy

cd mbbsd
make pmore
echo '0000000: 0c 23 00 2c 66 31 23 0a 30 ab ab ab ab ab ab     .#.,f1#.0......' | xxd -r > input
./pmore input

detected by afl-fuzz

When I was using the web search api with the following two requests, there something strange.
The results of the first request should include the one from the second result. (>=100 should fulfill >= 95)
However, the result doesn't fulfill this logic.

https://www.ptt.cc/bbs/Boy-Girl/search?q=recommend%3A95

爆
Re: [討論] 女生最討厭男網友問什麼？
FISHBONE0914
⋯
3/25
爆
Re: [討論] 女生最討厭男網友問什麼？
sumade
⋯
3/25
爆
[求助] 在一起沒多久就想分手
klyuch
⋯

https://www.ptt.cc/bbs/Boy-Girl/search?q=recommend%3A100

爆
Re: [討論] 這個影片是不是仇女???
yy10322
⋯
3/26
爆
Re: [討論] 所以你們有純友誼的女生朋友嗎？
beeeemo
⋯
3/26
爆
Re: [討論] 女生最討厭男網友問什麼？
FISHBONE0914
⋯
3/25
爆
Re: [討論] 女生最討厭男網友問什麼？
sumade
⋯

經過我的計算，此專案資料庫（P.T.T.B.B.S.）共有85個分流（Branch），其中31個來自於原作者（P.T.T.官方）。
因為這個數量有些多，所以希望官方能合併（Merge）一些分流。

就是post點歌排行榜顯示的名稱不會隨著BBSMNAME而改變
如果不想改太多的話是不是就把 [Ptt流行網] 改成 [點播流行網] 或其他不會用到BBS名字的名稱這樣?

mbbsd/bbs.c第817~824行有提供一個彩色日期的選項

    {
#ifdef COLORDATE
	prints(ANSI_COLOR(%d) "%-6.5s" ANSI_RESET,
		(ent->date[3] + ent->date[4]) % 7 + 31, ent->date);
#else
	prints("%-6.5s", ent->date);
#endif
    }

但預設是暗彩色,不知道是不是因為為了方便綠色日期跟推文數顏色區分?
不知道有沒有機會另外提供亮彩色選項 ANSI_COLOR(%d) 加上 1; 變成 ANSI_COLOR(1;%d)
還是需要時自己改code即可...orz

目前只有想到：(編譯測試後ok,會不會影響系統效能不知道...至少小站暫時看不出差別...

    {
#ifdef COLORDATE
    #ifdef COLORDATE_BRIGHT
        prints(ANSI_COLOR(1;%d) "%-6.5s" ANSI_RESET,
    #else
        prints(ANSI_COLOR(%d) "%-6.5s" ANSI_RESET,
    #endif
    (ent->date[3] + ent->date[4]) % 7 + 31, ent->date);
#else
    prints("%-6.5s", ent->date);
#endif
    }

可以自行精簡程式碼...(如果可以的話

Greetings, is there any sort of PTT dataset or API for accessing the comments similar to the comments dataset for reddit?

I want to conpile logind to support websocket. I follow the step in the this page. However I encounter this error every time. Do I miss some code or the wiki didn't update?

gcc -g -Os -W -Wall -Wunused -Wno-missing-field-initializers -pipe -DBBSHOME='"/home/bbs"' -I../../include  -Os -Wl,--as-needed -o logind logind.o ../../util/util_var.o  ../../common/bbs/libcmbbs.a  ../../common/sys/libcmsys.a  ../../common/osdep/libosdep.a  -levent
gcc: error: ../../util/util_var.o: No such file or directory
gcc: error: ../../common/bbs/libcmbbs.a: No such file or directory
gcc: error: ../../common/sys/libcmsys.a: No such file or directory
gcc: error: ../../common/osdep/libosdep.a: No such file or directory
*** Error code 1

Stop.
bmake: stopped in /home/bbs/pttbbs/daemon/login

.PASSWDS is one file recording all users' critical information.

In current setting (ssh/telnet/ws), we initialize userec in the login
to minimize reading the file.

However, it will be difficult for web-based service because
it's hard to keep track of userec without reading .PASSWD.
(.PASSWD is one gigantic file recording all the users.
It's not feasible to know whether data of one user is updated
just by looking at the mtime of the file.)

.PASSWDS is with fixed size / fixed columns and is easy to migrate to db-based (by uid and by userid) storage.

We can follow the typical setup for the smooth upgrade:

Read: try to get data from db first, if success: return data. otherwise, get data from file and write to db, return data.
Write: try both on file and db.

The followings are the files / functions using .PASSWDS (fn_passwd / FN_PASSWD):

util/mailangel.c: 57 (readData): fread(&user, sizeof(userec_t), 1, fp)
util/uhash_loader.c: 118 (fill_uhash): for (mimage = fimage; usernumber < fd; mimage += sizeof(userec_t)) (read)
util/tunepasswd.c: (當 MAX_USERS 長大時. 要把 .PASSWD 跟著長大. db 上可以不考慮這件事情～)
util/xchatd.c: 195: get_record(FN_PASSWD, acct, sizeof(ACCT), id)
util/angel.c: 70: fread(&user, sizeof(user), 1, fp)
daemon/regmaild/regmaild.c: 232: read(fd, &xuser, sizeof(xuser))
mbbsd/admin.c:397: fread(&user, sizeof(user), 1, fp1)

util/writemony.c (main)
common/bbs/passwd.c:
passwd_update_money
passwd_update
passwd_query

mbbsd/register.c:408 if ((fd = OpenCreate(fn_passwd, O_RDWR)) == -1)
(not exactly sure why it's there, looks like it just want to ensure that fn_passwd exists, but forgot to close fd)

The PR will include tests.

As Title

在 https://github.com/ptt/pttbbs/blob/master/include/pttstruct.h L215,216

#define BRD_VOTEBOARD		0x00000200	/* 連署機看板 */
#define BRD_WARNEL		0x00000400	/* 連署機看板 */

兩者是一樣的

然而在 https://github.com/ptt/pttbbs/blob/master/mbbsd/var.c L:88,89
註解為

    "連署專用看板",		/* BRD_VOTEBOARD */
    "已警告要廢除",		/* BRD_WARNEL */

因此擬修改 pttstruct.h 及 pttstruct.py :L140
連署機看板 為 已警告要廢除

I am interested in an English version of PTT, for use in primarily English speaking areas.

Has anyone done a translation? Or is there interest in doing a translation?

In Linux, a value at /proc/sys/kernel/pid_max handles how many processes it can contain.
On default, it is set to 32768.
But ptt handles over 10 thousands connections at a time.
It means ptt have to open over 10 thousands process at a time too. (If the connection is ssh)
I know there's a program called logind to handle connections in a single process.
I just wonder how ptt handle ssh connections...

能否新增使用者 IP 使用累計次數排名? 比方說, 假設有3個使用者 A, B, 和 C. 其中

A 使用的 IP 登入有 173.115.224.26, 211.76.137.43, 77.216.252.78, 77.90.103.139, 108.34.127.26.
B 過去使用登入的 IP 有 200.180.145.220, 126.2.91.185
C 登入的 IP 只有 13.53.68.254

依此排名顯示為

使用者 | 次數
    A     |    5 
    B     |    2 
    C     |    1 

排名可以依歷史累計(總共使用 IP 累計), 每天(或每個禮拜, 每個月)累計.

https://m.facebook.com/story.php?story_fbid=10208219475071485&id=1440581349&ref=bookmarks

看到今天中午公布的，早上才想到要用 HBase 來改寫，最近會開始分析 schema。

只要 rowkey 做的好，應該會讓整體效能變好。

這樣子擴充性應該會比較好，至少不用一直刪資料，不知道大家覺得如何？

Recently a popular / controversial board has seen unwanted changes to its board subtitle by one of its moderators. However no one claimed responsibility.

While the system currently logs modification actions, the log is not specific enough to say what setting was modified. It is a fluke that we might be able to discern changes to the subtitle by consecutive double log entries.

Going forward we should log any changes to board settings, with details on who did what exactly.
The log should be made available to current moderators of the board and moderators higher-up.
A log accessible via hot-key or menu, like the recycle bin or personal login/money logs would be nice.

The system currently does not allow users to freely modify their email address.

If they verified their account using an email address, then if they request a change,
the SYSOP would request they also re-verify their account.

If they verified their account with manual entered personal information, they
do not get to tie an email address to their account.

This whole setup is arcane and inconvenient. For example, it prevents us from
providing password reset services via email, among other things.

Check the code and see what needs to be changed, what might need to be
preserved across modifications, and where and how to log email changes.

https://github.com/ptt/pttbbs/blob/master/common/sys/string.c#L283

0xFFFD happened in u2b.
We can paste thai characters in utf8-terminal
and get 0xFFFD.

strip_nonebig5 affects
title in mbbsd/bbs.readdoent and
currutmp->nickname in setup_utmp.

An extra null pointer check is not needed in functions like the following.

• copyqueue_append
• substitute_record2
• Vector_resize

如使用者收通知要繳 1000元 罰款(等於死刑)起，一路未見到任何上訴，申訴，任何標語。請於軟體加之。如**國家警察得通知犯人有那些權利，能請律師等等。

目前遇到的問題：
沒有，只是想確認。

在 var.c:L397 當中的 SHM 理論上應該要加上 volatile 來保護？ 避免其他 Process 存取時因為編譯器優化所以讀到舊的值。

de63cdf broke wsproxy and it is generating a bunch of error messages like:

attempt to set status 400 via ngx.exit after sending out the response status 101

We need to use ngx.exit(444) after websocket connection has been established. This is a special code for nginx to terminate connection directly.

希望有API介面, 增加ptt的應用方式, 比方說開發者可透過ptt可公開的資料利用AI辨識fake news等等.