Comments (10)
nokchax
commented on August 17, 2024
1
- 페이지 응답속도가 1.5s 이내여야 사용자 이탈을 방지할 수 있을거 같습니다.
- 모든 것을 테스트해 보진 않았지만, 경로 검색에서 접속자 수가 많아질 수록 응답속도가 심각하게 느려져서, 이 부분에 캐시처리가 필요해 보입니다.
- 부하테스트 전제조건
- 로컬 테스트로 네트워크 이슈는 제외, 서버와 DB만 테스트한다.
- 사용자는 지하철 경로 검색을 주로 이용하며, 출퇴근 시간대에 요청이 몰린다.
- 한 사용자가 사용하는 기간은 1분 내외로 짧다.
- 최대 이용자 수는 2000명 내외다.
- 앱 서버는 DB외에 다른 시스템에는 연결되어 있지 않다. 고로 DB외의 다른 제약은 없다.
- 최대 이용자 수 내의 요청에서는 목표한 성능을 유지해야 한다.
- Smoke, Load, Stress 테스트 결과
메인페이지 접근 테스트
Smoke test
ubuntu@ip-192-5-10-38:~/test$ k6 run smoke.js
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: smoke.js
output: -
scenarios: (100.00%) 1 scenario, 1 max VUs, 40s max duration (incl. graceful stop):
* default: 1 looping VUs for 10s (gracefulStop: 30s)
running (10.2s), 0/1 VUs, 10 complete and 0 interrupted iterations
default ✓ [======================================] 1 VUs 10s
✓ logged in successfully
✓ retrieved member
checks.....................: 100.00% ✓ 20 ✗ 0
data_received..............: 4.9 kB 483 B/s
data_sent..................: 3.3 kB 321 B/s
http_req_blocked...........: avg=16.62µs min=4.14µs med=7.35µs max=206.61µs p(90)=10.01µs p(95)=24.23µs
http_req_connecting........: avg=6.95µs min=0s med=0s max=139.14µs p(90)=0s p(95)=6.95µs
✓ http_req_duration..........: avg=7.47ms min=3.7ms med=4.87ms max=51.05ms p(90)=6.84ms p(95)=14.91ms
http_req_failed............: 100.00% ✓ 20 ✗ 0
http_req_receiving.........: avg=327.65µs min=50.44µs med=310.86µs max=829.09µs p(90)=543.21µs p(95)=561.28µs
http_req_sending...........: avg=27.31µs min=12.47µs med=21.59µs max=80.44µs p(90)=46.83µs p(95)=51.79µs
http_req_tls_handshaking...: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...........: avg=7.11ms min=3.39ms med=4.48ms max=50.83ms p(90)=6.65ms p(95)=14.54ms
http_reqs..................: 20 1.969112/s
iteration_duration.........: avg=1.01s min=1s med=1.01s max=1.05s p(90)=1.02s p(95)=1.03s
iterations.................: 10 0.984556/s
vus........................: 1 min=1 max=1
vus_max....................: 1 min=1 max=1
Load test
ubuntu@ip-192-5-10-38:~/test$ k6 run load.js
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: load.js
output: -
scenarios: (100.00%) 1 scenario, 500 max VUs, 3m40s max duration (incl. graceful stop):
* default: Up to 500 looping VUs for 3m10s over 3 stages (gracefulRampDown: 30s, gracefulStop: 30s)
running (3m10.7s), 000/500 VUs, 76019 complete and 0 interrupted iterations
default ✓ [======================================] 000/500 VUs 3m10s
✓ logged in successfully
✓ retrieved member
checks.....................: 100.00% ✓ 152038 ✗ 0
data_received..............: 37 MB 196 kB/s
data_sent..................: 25 MB 130 kB/s
http_req_blocked...........: avg=30.15µs min=3µs med=4.72µs max=104.32ms p(90)=6.33µs p(95)=11.63µs
http_req_connecting........: avg=19.83µs min=0s med=0s max=103.75ms p(90)=0s p(95)=0s
✓ http_req_duration..........: avg=10.92ms min=643.78µs med=5.63ms max=458ms p(90)=24.26ms p(95)=39.5ms
http_req_failed............: 100.00% ✓ 152038 ✗ 0
http_req_receiving.........: avg=625.27µs min=17.11µs med=150.45µs max=134.25ms p(90)=1.18ms p(95)=2.28ms
http_req_sending...........: avg=124.64µs min=8.31µs med=15.83µs max=90.58ms p(90)=46.07µs p(95)=123.08µs
http_req_tls_handshaking...: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...........: avg=10.17ms min=500.03µs med=5.16ms max=457.95ms p(90)=22.77ms p(95)=37.16ms
http_reqs..................: 152038 797.271274/s
iteration_duration.........: avg=1.02s min=1s med=1.01s max=1.54s p(90)=1.04s p(95)=1.07s
iterations.................: 76019 398.635637/s
vus........................: 30 min=9 max=500
vus_max....................: 500 min=500 max=500
Stress test
사전에 /etc/security/limits.conf 에 아래와 같이 추가하여 파일 오픈 리밋을 풀어준다
ubuntu soft nofile 40960
ubuntu hard nofile 40960
export let options = {
stages: [
{ duration: '1m', target: 2000 },
{ duration: '10s', target: 0 },
],
thresholds: {
http_req_duration: ['p(99)<1500'], // 99% of requests must complete below 1.5s
'logged in successfully': ['p(99)<1500'], // 99% of requests must complete below 1.5s
},
};
ubuntu@ip-192-5-10-38:~/test$ k6 run stress.js
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: stress.js
output: -
scenarios: (100.00%) 1 scenario, 2000 max VUs, 1m40s max duration (incl. graceful stop):
* default: Up to 2000 looping VUs for 1m10s over 2 stages (gracefulRampDown: 30s, gracefulStop: 30s)
running (1m11.0s), 0000/2000 VUs, 43535 complete and 0 interrupted iterations
default ↓ [======================================] 0107/2000 VUs 1m10s
✓ logged in successfully
✓ retrieved member
checks.....................: 100.00% ✓ 87070 ✗ 0
data_received..............: 21 MB 301 kB/s
data_sent..................: 14 MB 201 kB/s
http_req_blocked...........: avg=751.09µs min=2.85µs med=4.57µs max=765.09ms p(90)=6.67µs p(95)=26.86µs
http_req_connecting........: avg=726.6µs min=0s med=0s max=762.85ms p(90)=0s p(95)=0s
✓ http_req_duration..........: avg=319.04ms min=585.52µs med=214.16ms max=2.02s p(90)=798.78ms p(95)=950.05ms
http_req_failed............: 100.00% ✓ 87070 ✗ 0
http_req_receiving.........: avg=2.11ms min=19.12µs med=59.49µs max=572.37ms p(90)=2ms p(95)=6.79ms
http_req_sending...........: avg=1.24ms min=8.77µs med=15.3µs max=311.64ms p(90)=156.67µs p(95)=668.39µs
http_req_tls_handshaking...: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...........: avg=315.68ms min=505.35µs med=211.76ms max=1.81s p(90)=792.88ms p(95)=937.26ms
http_reqs..................: 87070 1227.047237/s
iteration_duration.........: avg=1.64s min=1s med=1.51s max=4.16s p(90)=2.45s p(95)=2.78s
iterations.................: 43535 613.523619/s
vus........................: 107 min=34 max=2000
vus_max....................: 2000 min=2000 max=2000
2000명의 동접자 일때, 응답은 제대로 가지만 응답 속도가 약간 느려짐을 확인할 수 있음.
데이터 삽입 테스트 (가입 테스트)
Smoke test
import http from 'k6/http';
import { check, group, sleep, fail } from 'k6';
export let options = {
vus: 1, // 1 user looping for 1 minute
duration: '10s',
thresholds: {
http_req_duration: ['p(99)<1500'], // 99% of requests must complete below 1.5s
},
};
const BASE_URL = 'http://localhost:8080';
export default () => {
let joinRes = http.post(
`${BASE_URL}/members`,
JSON.stringify({
email: '[email protected]',
password: '1234',
age: 10,
}),
{ headers:{'Content-Type': 'application/json'} }
);
check(joinRes, {
'joined successfully': (resp) => resp.status === 201,
});
sleep(1);
};
ubuntu@ip-192-5-10-38:~/test/data-update$ k6 run smoke.js
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: smoke.js
output: -
scenarios: (100.00%) 1 scenario, 1 max VUs, 40s max duration (incl. graceful stop):
* default: 1 looping VUs for 10s (gracefulStop: 30s)
running (10.1s), 0/1 VUs, 10 complete and 0 interrupted iterations
default ✓ [======================================] 1 VUs 10s
✓ joined successfully
checks.........................: 100.00% ✓ 10 ✗ 0
data_received..................: 1.0 kB 98 B/s
data_sent......................: 2.0 kB 193 B/s
http_req_blocked...............: avg=66.91µs min=7.1µs med=7.36µs max=601.8µs p(90)=67.61µs p(95)=334.7µs
http_req_connecting............: avg=51.94µs min=0s med=0s max=519.4µs p(90)=51.94µs p(95)=285.67µs
✓ http_req_duration..............: avg=13.18ms min=7.2ms med=7.88ms max=60.24ms p(90)=14.28ms p(95)=37.26ms
{ expected_response:true }...: avg=13.18ms min=7.2ms med=7.88ms max=60.24ms p(90)=14.28ms p(95)=37.26ms
http_req_failed................: 0.00% ✓ 0 ✗ 10
http_req_receiving.............: avg=54.2µs min=43.63µs med=50.04µs max=80.48µs p(90)=64.57µs p(95)=72.52µs
http_req_sending...............: avg=48.71µs min=26.64µs med=36.89µs max=155.13µs p(90)=58.67µs p(95)=106.9µs
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...............: avg=13.08ms min=7.1ms med=7.8ms max=60.01ms p(90)=14.18ms p(95)=37.09ms
http_reqs......................: 10 0.986118/s
iteration_duration.............: avg=1.01s min=1s med=1s max=1.06s p(90)=1.01s p(95)=1.03s
iterations.....................: 10 0.986118/s
vus............................: 1 min=1 max=1
vus_max........................: 1 min=1 max=1
Load test
export let options = {
stages: [
{ duration: '1m', target: 500 },
{ duration: '2m', target: 500 },
{ duration: '10s', target: 0 },
],
thresholds: {
http_req_duration: ['p(99)<1500'], // 99% of requests must complete below 1.5s
'joined successfully': ['p(99)<1500'], // 99% of requests must complete below 1.5s
},
};
ubuntu@ip-192-5-10-38:~/test/data-update$ k6 run load.js
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: load.js
output: -
scenarios: (100.00%) 1 scenario, 500 max VUs, 3m40s max duration (incl. graceful stop):
* default: Up to 500 looping VUs for 3m10s over 3 stages (gracefulRampDown: 30s, gracefulStop: 30s)
running (3m10.8s), 000/500 VUs, 77199 complete and 0 interrupted iterations
default ✓ [======================================] 000/500 VUs 3m10s
✓ joined successfully
checks.........................: 100.00% ✓ 77199 ✗ 0
data_received..................: 7.7 MB 40 kB/s
data_sent......................: 15 MB 79 kB/s
http_req_blocked...............: avg=10.63µs min=3.52µs med=5.46µs max=17.77ms p(90)=6.84µs p(95)=10.49µs
http_req_connecting............: avg=3.01µs min=0s med=0s max=17.72ms p(90)=0s p(95)=0s
✓ http_req_duration..............: avg=6.84ms min=2.93ms med=5.82ms max=149.7ms p(90)=9.18ms p(95)=11.98ms
{ expected_response:true }...: avg=6.84ms min=2.93ms med=5.82ms max=149.7ms p(90)=9.18ms p(95)=11.98ms
http_req_failed................: 0.00% ✓ 0 ✗ 77199
http_req_receiving.............: avg=40.67µs min=11.86µs med=30.19µs max=13.69ms p(90)=52.06µs p(95)=67.82µs
http_req_sending...............: avg=32.12µs min=11.41µs med=17.53µs max=16.53ms p(90)=41.09µs p(95)=58.01µs
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...............: avg=6.77ms min=2.88ms med=5.75ms max=149.64ms p(90)=9.1ms p(95)=11.83ms
http_reqs......................: 77199 404.567287/s
iteration_duration.............: avg=1s min=1s med=1s max=1.15s p(90)=1s p(95)=1.01s
iterations.....................: 77199 404.567287/s
vus............................: 28 min=9 max=500
vus_max........................: 500 min=500 max=500
Stress test
export let options = {
stages: [
{ duration: '1m', target: 2000 },
{ duration: '10s', target: 0 },
],
thresholds: {
http_req_duration: ['p(99)<1500'], // 99% of requests must complete below 1.5s
'joined successfully': ['p(99)<1500'], // 99% of requests must complete below 1.5s
},
};
ubuntu@ip-192-5-10-38:~/test/data-update$ k6 run stress.js
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: stress.js
output: -
scenarios: (100.00%) 1 scenario, 2000 max VUs, 1m40s max duration (incl. graceful stop):
* default: Up to 2000 looping VUs for 1m10s over 2 stages (gracefulRampDown: 30s, gracefulStop: 30s)
running (1m11.0s), 0000/2000 VUs, 55970 complete and 0 interrupted iterations
default ✓ [======================================] 0000/2000 VUs 1m10s
✓ joined successfully
checks.........................: 100.00% ✓ 55970 ✗ 0
data_received..................: 5.6 MB 79 kB/s
data_sent......................: 11 MB 155 kB/s
http_req_blocked...............: avg=71.33µs min=3.46µs med=5.32µs max=322.87ms p(90)=8.3µs p(95)=30.33µs
http_req_connecting............: avg=48.06µs min=0s med=0s max=203.36ms p(90)=0s p(95)=0s
✓ http_req_duration..............: avg=273.6ms min=2.96ms med=135.41ms max=1.75s p(90)=712.43ms p(95)=839.61ms
{ expected_response:true }...: avg=273.6ms min=2.96ms med=135.41ms max=1.75s p(90)=712.43ms p(95)=839.61ms
http_req_failed................: 0.00% ✓ 0 ✗ 55970
http_req_receiving.............: avg=208.81µs min=12.6µs med=30.07µs max=161.48ms p(90)=61.9µs p(95)=174.65µs
http_req_sending...............: avg=468.34µs min=11.57µs med=17.06µs max=340.54ms p(90)=61µs p(95)=153.27µs
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...............: avg=272.93ms min=2.92ms med=134.88ms max=1.75s p(90)=711.61ms p(95)=838.44ms
http_reqs......................: 55970 788.843855/s
iteration_duration.............: avg=1.27s min=1s med=1.13s max=2.75s p(90)=1.71s p(95)=1.84s
iterations.....................: 55970 788.843855/s
vus............................: 111 min=34 max=1999
vus_max........................: 2000 min=2000 max=2000
응답속도 느려짐..
단순 가입자에 이렇게 몰릴 일이 없으니, 테스트 설정이 잘못된 듯..
여러 데이터 조회 테스트
Smoke test
import http from 'k6/http';
import { check, group, sleep, fail } from 'k6';
export let options = {
vus: 1, // 1 user looping for 1 minute
duration: '10s',
thresholds: {
http_req_duration: ['p(99)<1500'], // 99% of requests must complete below 1.5s
},
};
const BASE_URL = 'http://localhost:8080';
export default () => {
let inquireRes = http.get(`${BASE_URL}/paths?source=1&target=4`);
check(inquireRes, {
'inquire successfully': (resp) => resp.json('stations') !== undefined,
});
sleep(1);
};
ubuntu@ip-192-5-10-38:~/test/inquire$ k6 run smoke.js
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: smoke.js
output: -
scenarios: (100.00%) 1 scenario, 1 max VUs, 40s max duration (incl. graceful stop):
* default: 1 looping VUs for 10s (gracefulStop: 30s)
running (10.3s), 0/1 VUs, 10 complete and 0 interrupted iterations
default ✓ [======================================] 1 VUs 10s
✓ inquire successfully
checks.........................: 100.00% ✓ 10 ✗ 0
data_received..................: 6.0 kB 584 B/s
data_sent......................: 1.0 kB 99 B/s
http_req_blocked...............: avg=31.96µs min=6.67µs med=7.1µs max=253.11µs p(90)=34.08µs p(95)=143.6µs
http_req_connecting............: avg=18.28µs min=0s med=0s max=182.83µs p(90)=18.28µs p(95)=100.56µs
✓ http_req_duration..............: avg=32.99ms min=28.63ms med=32.02ms max=41.77ms p(90)=39.36ms p(95)=40.57ms
{ expected_response:true }...: avg=32.99ms min=28.63ms med=32.02ms max=41.77ms p(90)=39.36ms p(95)=40.57ms
http_req_failed................: 0.00% ✓ 0 ✗ 10
http_req_receiving.............: avg=706.19µs min=158.29µs med=399.95µs max=3.41ms p(90)=912.13µs p(95)=2.16ms
http_req_sending...............: avg=31.15µs min=19.37µs med=24.39µs max=74.16µs p(90)=44.43µs p(95)=59.29µs
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...............: avg=32.25ms min=28.29ms med=31.59ms max=38.39ms p(90)=38.34ms p(95)=38.36ms
http_reqs......................: 10 0.967478/s
iteration_duration.............: avg=1.03s min=1.02s med=1.03s max=1.04s p(90)=1.04s p(95)=1.04s
iterations.....................: 10 0.967478/s
vus............................: 1 min=1 max=1
vus_max........................: 1 min=1 max=1
Load test
export let options = {
stages: [
{ duration: '1m', target: 500 },
{ duration: '2m', target: 500 },
{ duration: '10s', target: 0 },
],
thresholds: {
http_req_duration: ['p(99)<1500'], // 99% of requests must complete below 1.5s
'inquire successfully': ['p(99)<1500'], // 99% of requests must complete below 1.5s
},
};
ubuntu@ip-192-5-10-38:~/test/inquire$ k6 run load.js
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: load.js
output: -
scenarios: (100.00%) 1 scenario, 500 max VUs, 3m40s max duration (incl. graceful stop):
* default: Up to 500 looping VUs for 3m10s over 3 stages (gracefulRampDown: 30s, gracefulStop: 30s)
running (3m10.9s), 000/500 VUs, 26181 complete and 0 interrupted iterations
default ✓ [======================================] 000/500 VUs 3m10s
✓ inquire successfully
checks.........................: 100.00% ✓ 26181 ✗ 0
data_received..................: 16 MB 83 kB/s
data_sent......................: 2.7 MB 14 kB/s
http_req_blocked...............: avg=11.45µs min=3.6µs med=5.68µs max=12.56ms p(90)=7.22µs p(95)=11.03µs
http_req_connecting............: avg=3.7µs min=0s med=0s max=12.5ms p(90)=0s p(95)=0s
✗ http_req_duration..............: avg=1.98s min=24.34ms med=2.35s max=5.11s p(90)=2.45s p(95)=2.51s
{ expected_response:true }...: avg=1.98s min=24.34ms med=2.35s max=5.11s p(90)=2.45s p(95)=2.51s
http_req_failed................: 0.00% ✓ 0 ✗ 26181
http_req_receiving.............: avg=1.6ms min=24.82µs med=608.51µs max=74.63ms p(90)=3.51ms p(95)=5.97ms
http_req_sending...............: avg=21.49µs min=9.63µs med=15.44µs max=9.07ms p(90)=30.24µs p(95)=50.35µs
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...............: avg=1.97s min=24.02ms med=2.35s max=5.11s p(90)=2.45s p(95)=2.51s
http_reqs......................: 26181 137.124818/s
iteration_duration.............: avg=2.98s min=1.02s med=3.35s max=6.11s p(90)=3.45s p(95)=3.51s
iterations.....................: 26181 137.124818/s
vus............................: 23 min=9 max=500
vus_max........................: 500 min=500 max=500
ERRO[0192] some thresholds have failed
응답시간이 많이 느려짐
Stress test
export let options = {
stages: [
{ duration: '1m', target: 2000 },
{ duration: '10s', target: 0 },
],
thresholds: {
http_req_duration: ['p(99)<1500'], // 99% of requests must complete below 1.5s
'inquire successfully': ['p(99)<1500'], // 99% of requests must complete below 1.5s
},
};
ubuntu@ip-192-5-10-38:~/test/inquire$ k6 run stress.js
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: stress.js
output: -
scenarios: (100.00%) 1 scenario, 2000 max VUs, 1m40s max duration (incl. graceful stop):
* default: Up to 2000 looping VUs for 1m10s over 2 stages (gracefulRampDown: 30s, gracefulStop: 30s)
running (1m20.0s), 0000/2000 VUs, 11064 complete and 0 interrupted iterations
default ↓ [======================================] 1352/2000 VUs 1m10s
✓ inquire successfully
checks.........................: 100.00% ✓ 11064 ✗ 0
data_received..................: 6.7 MB 84 kB/s
data_sent......................: 1.1 MB 14 kB/s
http_req_blocked...............: avg=55.69µs min=3.65µs med=5.83µs max=21.34ms p(90)=200.5µs p(95)=218.87µs
http_req_connecting............: avg=37.42µs min=0s med=0s max=21.27ms p(90)=135.64µs p(95)=147.88µs
✗ http_req_duration..............: avg=6.68s min=23.95ms med=6.85s max=16.35s p(90)=12.56s p(95)=13.52s
{ expected_response:true }...: avg=6.68s min=23.95ms med=6.85s max=16.35s p(90)=12.56s p(95)=13.52s
http_req_failed................: 0.00% ✓ 0 ✗ 11064
http_req_receiving.............: avg=1.54ms min=25.96µs med=610.51µs max=220.77ms p(90)=3.31ms p(95)=5.39ms
http_req_sending...............: avg=28.79µs min=10.64µs med=16.3µs max=8.32ms p(90)=59.86µs p(95)=75.15µs
http_req_tls_handshaking.......: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...............: avg=6.68s min=23.83ms med=6.85s max=16.35s p(90)=12.56s p(95)=13.52s
http_reqs......................: 11064 138.254081/s
iteration_duration.............: avg=7.69s min=1.02s med=7.85s max=17.35s p(90)=13.63s p(95)=14.52s
iterations.....................: 11064 138.254081/s
vus............................: 8 min=8 max=2000
vus_max........................: 2000 min=2000 max=2000
ERRO[0082] some thresholds have failed
응답속도 16초 까지 나옴
응답속도 향상을 위해 캐시처리가 필요함.
테스트 자체는 그렇게 어렵지 않은데, 테스트 전제 조건을 어떻게 설정하고, 테스트 시 동시 접속자 수를 어느 정도로 설정해야 하는지 정하는게 어려웠습니다. 질문에 대한 답을 맞게 했는지도 잘 모르겠네요.
그리고 테스트 완료한 후에 보니까, 데이터 갱신에 대한 테스트 시나리오를 잘못 짠거 같아서, 시간이 남으면 회원정보를 수정하는 식으로 수정해보려고 합니다. 감사합니다 :)
from infra-subway.
woowahanCU
commented on August 17, 2024
1
-
동시접속자 수를 어느정도로 설정해야 하는지
우선, 목푯값 설정할 때 동접자(Concurrent User)와 Active User를 구분해서 생각하면 좋을거 같아요.
Active User로 정하기 위해서는, 우선 rps를 계산해보아야 하는데요.
rps 계산에 대해서는 지난 수업시간에 다룬 부분을 참고해보시면 좋을거 같아요.
목표 throughput과 목표 latency를 구하고 나면,
몇명의 VUser일 때 rps(k6에서는 http_reqs)가 목표값에 이르는지 테스트해보고 맞춰보면 될거 같아요. -
limits.conf 설정을 수정하셨군요~ 👍🏻
from infra-subway.
fistkim101
commented on August 17, 2024
1
1) 웹 성능예산은 어느정도가 적당하다고 생각하시나요
웹 성능예산이 무엇인지에 대한 내용은 강의에서도 배웠고 이에 관한 내용들도 많은데 어떤 기준으로 성능예산을 잡아야할지에 대해서는 정리된 글을 찾기가 어려웠습니다.
그러던 중 이 사이트 에서 관련된 부분이 있어서 참고하였습니다.
서비스 하고자 하는 사이트와 유사한 (경쟁)사이트들에 대해서 조사한 뒤 성능을 점검하고,
서비스 하고자 하는 사이트의 규모와 특성을 감안하여 (경쟁)사이트들에 비해 뒤쳐지지 않도록 웹 성능예산을 정하되
정말 정하기가 모호하다면 일반적인 성능예산(로드타임 3초 이내, TTI 5초 이내 등 강의 자료에도 나온 여러 예시들) 등을 기준으로 잡으면 좋지 않을까 생각해봤습니다.
2) 웹 성능예산을 바탕으로 현재 지하철 노선도 서비스는 어떤 부분을 개선하면 좋을까요
pagespeed 에서 추천 해준 것중 예상 절감치가 가장 컸던 '텍스트 압축 사용'을 위해서
.properties 설정 내 server.compression.enabled, server.compression.mime-types를 추가해줬더니
점수가 21점에서 40점으로 올라갔습니다.
.properties 내 설정
- server.compression.enabled=true
- server.compression.mime-types=application/json,application/xml,text/html,text/xml,text/plain,application/javascript,text/css
3) 부하테스트 전제조건은 어느정도로 설정하셨나요
목표 rps를 얼마로 잡아야 할지 잘 모르겠어서, 목표 latency만 '100ms 이하'로 잡고 몇 VUser일 때 rps가 얼마가 나오는지를
VUser수를 바꿔가면서 체크해봤습니다.
한계치를 아래와 같이 설정한 상태에서 VUser를 늘려가며 테스트했습니다.
thresholds: { http_req_duration: ['p(95)<100'], },
4) Smoke, Load, Stress 테스트 결과를 공유해주세요
lending page 접속 테스트
lengding page 방문 -> 회원가입 테스트
95퍼센트 이상의 요청에 대해서 latency를 100ms 이내로 커버하도록 한계점을 설정해놨더니,
생각보다 VUser가 얼마 되지 않는데도 한계점에 대부분 걸렸습니다.
사용한 테스트 스크립트는 아래와 같습니다.(load 기준)
lending
lending -> 회원가입
감사합니다.
from infra-subway.
mook1594
commented on August 17, 2024
1
1.
정량 기반 지표
- 이미지 파일크기 < 2MB
- 스크립트 파일 크기 < 1MB
- 웹 폰트 파일 개수
시간 기반 지표
- 전체 로딩 시간 < 3초
- FCP < 1초
- TTI < 2초
규칙 기반 지표
- 성능 점수 lighthouse > 80
2. Lighthouse 지하철 노선도 지표
현재 지하철 노선도 지표
- FCP = 4.682s
- TTI = 5.226s
- 스크립트 파일 크기 /js/vendors.js - 2,125.0 KiB
- 캐싱 정책 사용 필요
- 폰트 woff 사용 필요
- 스크립트 파일 정리 필요
3.
대상 시스템 범위
목표값 설정
- 1일 예상 총 이용자수: 200,000명 (가정)
- 1명당 1일 평균 요청수: 15
- 1일 총 접속 수 : 일 사용자 수 (200,000) x 1일 평균 접속 수 (15) = 3,000,000
- 1일 평균 rps: 1일 총 접속 수 (3,000,000) / 64,800 (새벽시간에 안쓰는걸 감안하여 5시~11시로 가정) = 46.3 rps
- 최대 트래픽 / 평균 트래픽: 4 배 (가정)
- 1일 최대 rps: 1일 평균 rps (46.3) x 4 = 185.2 rps
부하테스트
- 접속 빈도가 높은 페이지: 경로 검색 페이지
- 데이터를 갱신하는 페이지: 회원가입
- 데이터를 조회하는데 여러 데이터를 참조하는 페이지: 경로 탐색 로직 동작 호출
4.
Smoke Test
데이터를 갱신하는 페이지
checks.........................: 100.00% ✓ 20 ✗ 0
data_received..................: 18 kB 1.7 kB/s
data_sent......................: 3.8 kB 373 B/s
http_req_blocked...............: avg=2.17ms min=4.47µs med=6.77µs max=43.42ms p(90)=10.64µs p(95)=2.18ms
http_req_connecting............: avg=75.22µs min=0s med=0s max=1.5ms p(90)=0s p(95)=75.22µs
✓ http_req_duration..............: avg=11.52ms min=2.07ms med=8.74ms max=58.18ms p(90)=18.81ms p(95)=21.39ms
{ expected_response:true }...: avg=11.52ms min=2.07ms med=8.74ms max=58.18ms p(90)=18.81ms p(95)=21.39ms
http_req_failed................: 0.00% ✓ 0 ✗ 20
http_req_receiving.............: avg=72.25µs min=41.85µs med=67.7µs max=110.72µs p(90)=101.26µs p(95)=104.5µs
http_req_sending...............: avg=31.61µs min=18.79µs med=23.25µs max=113.46µs p(90)=48.5µs p(95)=52.07µs
http_req_tls_handshaking.......: avg=1.61ms min=0s med=0s max=32.28ms p(90)=0s p(95)=1.61ms
http_req_waiting...............: avg=11.41ms min=1.95ms med=8.66ms max=58.08ms p(90)=18.72ms p(95)=21.3ms
http_reqs......................: 10 1.945149/s
iteration_duration.............: avg=1.02s min=1.01s med=1.01s max=1.1s p(90)=1.03s p(95)=1.06s
iterations.....................: 10 0.972575/s
vus............................: 1 min=1 max=1
vus_max........................: 1 min=1 max=1
데이터를 조회하는데 여러 데이터를 참조하는 페이지
checks.........................: 100.00% ✓ 10 ✗ 0
data_received..................: 8.1 kB 723 B/s
data_sent......................: 1.7 kB 154 B/s
http_req_blocked...............: avg=6.27ms min=7.76µs med=8.36µs max=62.68ms p(90)=6.29ms p(95)=34.48ms
http_req_connecting............: avg=895.31µs min=0s med=0s max=8.95ms p(90)=895.31µs p(95)=4.92ms
✓ http_req_duration..............: avg=107.07ms min=67.31ms med=89.57ms max=170.99ms p(90)=167ms p(95)=168.99ms
{ expected_response:true }...: avg=107.07ms min=67.31ms med=89.57ms max=170.99ms p(90)=167ms p(95)=168.99ms
http_req_failed................: 0.00% ✓ 0 ✗ 10
http_req_receiving.............: avg=97.94µs min=76.84µs med=100.08µs max=115.04µs p(90)=114.07µs p(95)=114.56µs
http_req_sending...............: avg=39.1µs min=26.7µs med=33.41µs max=83.79µs p(90)=48.48µs p(95)=66.13µs
http_req_tls_handshaking.......: avg=4.47ms min=0s med=0s max=44.78ms p(90)=4.47ms p(95)=24.63ms
http_req_waiting...............: avg=106.93ms min=67.18ms med=89.44ms max=170.85ms p(90)=166.82ms p(95)=168.84ms
http_reqs......................: 10 0.897677/s
iteration_duration.............: avg=1.11s min=1.06s med=1.09s max=1.22s p(90)=1.17s p(95)=1.2s
iterations.....................: 10 0.897677/s
vus............................: 1 min=1 max=1
vus_max........................: 1 min=1 max=1
접속 빈도가 높은 페이지
checks.........................: 100.00% ✓ 10 ✗ 0
data_received..................: 16 kB 1.5 kB/s
data_sent......................: 1.5 kB 150 B/s
http_req_blocked...............: avg=5.72ms min=7.73µs med=8.2µs max=57.18ms p(90)=5.72ms p(95)=31.45ms
http_req_connecting............: avg=676.63µs min=0s med=0s max=6.76ms p(90)=676.63µs p(95)=3.72ms
✓ http_req_duration..............: avg=3.25ms min=2.04ms med=2.19ms max=10.18ms p(90)=5.6ms p(95)=7.89ms
{ expected_response:true }...: avg=3.25ms min=2.04ms med=2.19ms max=10.18ms p(90)=5.6ms p(95)=7.89ms
http_req_failed................: 0.00% ✓ 0 ✗ 10
http_req_receiving.............: avg=76.9µs min=57.64µs med=72.2µs max=111.28µs p(90)=104.04µs p(95)=107.66µs
http_req_sending...............: avg=37.08µs min=23.18µs med=34.15µs max=83.93µs p(90)=43.97µs p(95)=63.95µs
http_req_tls_handshaking.......: avg=4.96ms min=0s med=0s max=49.67ms p(90)=4.96ms p(95)=27.32ms
http_req_waiting...............: avg=3.14ms min=1.93ms med=2.09ms max=9.99ms p(90)=5.48ms p(95)=7.73ms
http_reqs......................: 10 0.990499/s
iteration_duration.............: avg=1s min=1s med=1s max=1.06s p(90)=1.01s p(95)=1.03s
iterations.....................: 10 0.990499/s
vus............................: 1 min=1 max=1
vus_max........................: 1 min=1 max=1
Load Test
stages: [
{ duration: '10s', target: 50 },
{ duration: '1m', target: 50 },
{ duration: '30s', target: 200 },
{ duration: '1m', target: 200 },
{ duration: '30s', target: 50 },
{ duration: '1m', target: 50 },
{ duration: '10s', target: 0 },
],
thresholds: {
http_req_duration: ['p(99)<1500'], // 99% of requests must complete below 1.5s
'logged in successfully': ['p(99)<1500'], // 99% of requests must complete below 1.5s
},
- 목표값으로 설정한 일평균 rps, 최대 rps 일때 성능 테스트.
- vus별 1초에 한번 request 하게 했을때 1초에 rps = vus 기반으로 target 구성
- 일평균 rps 로 1분 유지 -> 일최대rps 로 1분유지 -> 다시 일평균 rps로 돌아오는 시나리오
데이터를 갱신하는 페이지
checks.........................: 100.00% ✓ 66754 ✗ 0
data_received..................: 52 MB 198 kB/s
data_sent......................: 12 MB 46 kB/s
http_req_blocked...............: avg=91.57µs min=3.31µs med=5.14µs max=146.92ms p(90)=7.21µs p(95)=9.36µs
http_req_connecting............: avg=13.4µs min=0s med=0s max=50.13ms p(90)=0s p(95)=0s
✓ http_req_duration..............: avg=10.58ms min=1.17ms med=9.82ms max=208.62ms p(90)=18.01ms p(95)=24.79ms
{ expected_response:true }...: avg=10.58ms min=1.17ms med=9.82ms max=208.62ms p(90)=18.01ms p(95)=24.79ms
http_req_failed................: 0.00% ✓ 0 ✗ 66754
http_req_receiving.............: avg=51.91µs min=16.16µs med=44.57µs max=9.66ms p(90)=70.58µs p(95)=81.82µs
http_req_sending...............: avg=24.24µs min=9.91µs med=16.6µs max=8.69ms p(90)=34µs p(95)=50.67µs
http_req_tls_handshaking.......: avg=70.68µs min=0s med=0s max=122.19ms p(90)=0s p(95)=0s
http_req_waiting...............: avg=10.51ms min=1.12ms med=9.76ms max=208.56ms p(90)=17.94ms p(95)=24.7ms
http_reqs......................: 66754 256.185188/s
iteration_duration.............: avg=1.02s min=1s med=1.01s max=1.4s p(90)=1.02s p(95)=1.04s
iterations.....................: 33377 128.092594/s
vus............................: 5 min=5 max=200
vus_max........................: 200 min=200 max=200
데이터를 조회하는데 여러 데이터를 참조하는 페이지
checks.........................: 100.00% ✓ 19582 ✗ 0
data_received..................: 11 MB 42 kB/s
data_sent......................: 2.7 MB 10 kB/s
http_req_blocked...............: avg=125.41µs min=3.92µs med=6.77µs max=61.15ms p(90)=10.24µs p(95)=30.51µs
http_req_connecting............: avg=23.1µs min=0s med=0s max=7.93ms p(90)=0s p(95)=0s
✗ http_req_duration..............: avg=744.48ms min=55.36ms med=365.3ms max=5.8s p(90)=1.62s p(95)=1.69s
{ expected_response:true }...: avg=744.48ms min=55.36ms med=365.3ms max=5.8s p(90)=1.62s p(95)=1.69s
http_req_failed................: 0.00% ✓ 0 ✗ 19582
http_req_receiving.............: avg=74.51µs min=26.15µs med=69.01µs max=4.45ms p(90)=104.08µs p(95)=118.56µs
http_req_sending...............: avg=29.98µs min=10.78µs med=20.45µs max=15.06ms p(90)=53.36µs p(95)=70.69µs
http_req_tls_handshaking.......: avg=90.62µs min=0s med=0s max=35.94ms p(90)=0s p(95)=0s
http_req_waiting...............: avg=744.38ms min=55.2ms med=365.19ms max=5.8s p(90)=1.62s p(95)=1.69s
http_reqs......................: 19582 75.077614/s
iteration_duration.............: avg=1.74s min=1.05s med=1.36s max=6.81s p(90)=2.62s p(95)=2.69s
iterations.....................: 19582 75.077614/s
vus............................: 7 min=7 max=200
vus_max........................: 200 min=200 max=200
=> 최대 vus가 높아질수록 느려져 약 6초로 개선의 여지가 있음
접속 빈도가 높은 페이지
checks.........................: 100.00% ✓ 33996 ✗ 0
data_received..................: 44 MB 169 kB/s
data_sent......................: 4.0 MB 16 kB/s
http_req_blocked...............: avg=75.51µs min=3.75µs med=6.09µs max=40.09ms p(90)=8.77µs p(95)=13.91µs
http_req_connecting............: avg=11.75µs min=0s med=0s max=6.47ms p(90)=0s p(95)=0s
✓ http_req_duration..............: avg=2.76ms min=1.14ms med=2.36ms max=56.26ms p(90)=4.16ms p(95)=5.16ms
{ expected_response:true }...: avg=2.76ms min=1.14ms med=2.36ms max=56.26ms p(90)=4.16ms p(95)=5.16ms
http_req_failed................: 0.00% ✓ 0 ✗ 33996
http_req_receiving.............: avg=59.06µs min=21.13µs med=53.58µs max=4.94ms p(90)=78.19µs p(95)=90.41µs
http_req_sending...............: avg=24.47µs min=10.45µs med=16.4µs max=5.1ms p(90)=36.9µs p(95)=51.9µs
http_req_tls_handshaking.......: avg=54.9µs min=0s med=0s max=31.69ms p(90)=0s p(95)=0s
http_req_waiting...............: avg=2.68ms min=1.09ms med=2.27ms max=56.19ms p(90)=4.07ms p(95)=5.08ms
http_reqs......................: 33996 130.324456/s
iteration_duration.............: avg=1s min=1s med=1s max=1.05s p(90)=1s p(95)=1s
iterations.....................: 33996 130.324456/s
vus............................: 6 min=6 max=200
vus_max........................: 200 min=200 max=200
Stress Test
stages: [
{ duration: '10s', target: 100 },
{ duration: '30s', target: 100 },
{ duration: '30s', target: 200 },
{ duration: '30s', target: 200 },
{ duration: '30s', target: 300 },
{ duration: '30s', target: 300 },
{ duration: '30s', target: 400 },
{ duration: '30s', target: 400 },
{ duration: '10s', target: 0 },
],
thresholds: {
http_req_duration: ['p(99)<1500'], // 99% of requests must complete below 1.5s
'logged in successfully': ['p(99)<1500'], // 99% of requests must complete below 1.5s
},
- 목표값으로 설정한 일평균 rps, 최대 rps 일때 성능 테스트.
- vus별 스크립트에서 1초에 한번 request 하게 했을때 1초에 rps = vu 기반으로 target 구성
- vus 100씩 증가하도록 구성, 최대 일최대 rps 보다 더 커졌을때 성능 테스트 구성
데이터를 갱신하는 페이지
checks.........................: 99.96% ✓ 91339 ✗ 29
data_received..................: 89 MB 385 kB/s
data_sent......................: 19 MB 82 kB/s
http_req_blocked...............: avg=9.69ms min=0s med=5.05µs max=752.07ms p(90)=22.75µs p(95)=51.3ms
http_req_connecting............: avg=503.55µs min=0s med=0s max=194.21ms p(90)=0s p(95)=1.21ms
✓ http_req_duration..............: avg=96.73ms min=0s med=24.03ms max=1m0s p(90)=303.33ms p(95)=423.89ms
{ expected_response:true }...: avg=96.09ms min=1.27ms med=24.03ms max=1.48s p(90)=303.39ms p(95)=423.9ms
http_req_failed................: 0.03% ✓ 29 ✗ 91339
http_req_receiving.............: avg=105.58µs min=0s med=42.21µs max=170.23ms p(90)=74.75µs p(95)=104.95µs
http_req_sending...............: avg=4.09ms min=0s med=16.94µs max=830.02ms p(90)=72.27µs p(95)=221.89µs
http_req_tls_handshaking.......: avg=6.11ms min=0s med=0s max=550.25ms p(90)=0s p(95)=18.62ms
http_req_waiting...............: avg=92.52ms min=0s med=23.9ms max=1m0s p(90)=286.11ms p(95)=406.35ms
http_reqs......................: 91368 395.635877/s
iteration_duration.............: avg=1.2s min=1s med=1.05s max=1m1s p(90)=1.65s p(95)=1.78s
iterations.....................: 45684 197.817939/s
vus............................: 21 min=10 max=400
vus_max........................: 400 min=400 max=400
=> 최대 rps를 넘어서고 부하가 계속 증가할 수록 request 실패하는 케이스 발생
데이터를 조회하는데 여러 데이터를 참조하는 페이지
checks.........................: 100.00% ✓ 12011 ✗ 0
data_received..................: 65 MB 282 kB/s
data_sent......................: 28 MB 121 kB/s
http_req_blocked...............: avg=23.47ms min=0s med=0s max=1.25s p(90)=92.5ms p(95)=150.77ms
http_req_connecting............: avg=26.07ms min=0s med=17.9ms max=1.2s p(90)=57.84ms p(95)=79.17ms
✗ http_req_duration..............: avg=393.02ms min=0s med=0s max=24.34s p(90)=538.06ms p(95)=1.91s
{ expected_response:true }...: avg=2.93s min=52.44ms med=1.69s max=24.34s p(90)=9.15s p(95)=9.37s
http_req_failed................: 86.65% ✓ 77967 ✗ 12011
http_req_receiving.............: avg=200.76µs min=0s med=0s max=242.22ms p(90)=56.05µs p(95)=82.11µs
http_req_sending...............: avg=167.91µs min=0s med=0s max=111.19ms p(90)=63.18µs p(95)=560.24µs
http_req_tls_handshaking.......: avg=18.91ms min=0s med=0s max=676.31ms p(90)=76.05ms p(95)=120.31ms
http_req_waiting...............: avg=392.65ms min=-161.432µs med=0s max=24.32s p(90)=537.97ms p(95)=1.91s
http_reqs......................: 89978 389.61087/s
iteration_duration.............: avg=616.76ms min=2.77ms med=85.88ms max=25.45s p(90)=1.53s p(95)=2.91s
iterations.....................: 89978 389.61087/s
vus............................: 77 min=10 max=400
vus_max........................: 400 min=400 max=400
=> 최대 rps를 넘어서고 부하가 계속 증가할 수록 속도가 심하게 느려지고, request 실패하는 케이스 발생
접속 빈도가 높은 페이지
checks.........................: 100.00% ✓ 54968 ✗ 0
data_received..................: 72 MB 310 kB/s
data_sent......................: 6.6 MB 28 kB/s
http_req_blocked...............: avg=105.13µs min=3.54µs med=5.94µs max=51.75ms p(90)=8.02µs p(95)=10.82µs
http_req_connecting............: avg=21.67µs min=0s med=0s max=20.03ms p(90)=0s p(95)=0s
✓ http_req_duration..............: avg=3.08ms min=1.18ms med=2.57ms max=35.44ms p(90)=5.01ms p(95)=6.1ms
{ expected_response:true }...: avg=3.08ms min=1.18ms med=2.57ms max=35.44ms p(90)=5.01ms p(95)=6.1ms
http_req_failed................: 0.00% ✓ 0 ✗ 54968
http_req_receiving.............: avg=57.78µs min=20.38µs med=51.1µs max=6.97ms p(90)=75.8µs p(95)=89.05µs
http_req_sending...............: avg=23.98µs min=9.85µs med=15.81µs max=8.29ms p(90)=35.66µs p(95)=56.4µs
http_req_tls_handshaking.......: avg=75.01µs min=0s med=0s max=36.91ms p(90)=0s p(95)=0s
http_req_waiting...............: avg=2.99ms min=1.12ms med=2.49ms max=35.4ms p(90)=4.93ms p(95)=6.02ms
http_reqs......................: 54968 238.090315/s
iteration_duration.............: avg=1s min=1s med=1s max=1.05s p(90)=1s p(95)=1s
iterations.....................: 54968 238.090315/s
vus............................: 22 min=10 max=400
vus_max........................: 400 min=400 max=400
최대한 요구 사항에 맞게 예산과 전제조건 등을 파악하여
시나리오에 맞춰 성능 테스트를 해봤는데
흐름대로 잘 된건지 모르겠네요 :)
from infra-subway.
sk1737030
commented on August 17, 2024
1
-
웹 성능예산은 어느정도가 적당하다고 생각하시나요
- 페이지 로드시 1.5s 내외 정도로 응답이 되어야 한다.
- Lighthouse 성능 검사에서 80 점 이상이어야 한다.
-
웹 성능예산을 바탕으로 현재 지하철 노선도 서비스는 어떤 부분을 개선하면 좋을까요
- 중요하지 않는 자원(css, js, image 등)을 지연로딩
- 정적 컨텐츠 캐싱하기
- Compress Transfer 적용
-
부하테스트 전제조건은 어느정도로 설정하셨나요
- 예상 1일 사용자 수 1일 총 접속 수: 200,000만
- 1명당 1일 평균 접속: 10번
- 1일 총 접속 수: 2,000,000 번
- 1일 평균 rps(00~05시 제외): 30rps
- 1일 최대 rps는 10배 : 300rps
- 응답은 1.5s 이하로 응답
- 에러률 1% 미만
부하테스트를 위한 페이지
- 접속 빈도가 높은 페이지: 메인페이지
- 데이터를 조회하는데 여러 데이터를 참조하는 페이지: 경로검색페이지
- 데이터를 갱신하는 페이지 : 역 관리 페이지에 지하철역 추가
- Smoke, Load, Stress 테스트 결과를 공유해주세요
Smoke Test JS
export let options = {
vus: 1,
duration: '10s',
thresholds: {
http_req_duration: ['p(99)<1500'], // 99% of requests must complete below 1.5s
http_req_faild: ['rate<0.01'], // error should be less than 1%
}
};
const BASE_URL = 'https://dongguri.kro.kr/';
export default () => {
// 데이터를 조회하는데 여러 데이터를 참조하는 페이지:
let searchPath = http.get(`${BASE_URL}/paths?source=2&target=1`);
check(searchPath,
{ 'search paths 200': (resp) => resp.status === 200 },
{ tags: { type: "test" } },
);
// 데이터를 갱신하는 페이지
var randomName = Math.floor(Math.random() * 10000000000).toString();
let addStation = http.post(`${BASE_URL}/stations`, JSON.stringify({ name: randomName }), { headers: { 'Content-Type': 'application/json' } });
check(addStation, { 'add_station_paths_201': (resp) => resp.status === 201 });
// 접속 빈도가 높은 페이지
let accessPage = http.get(`${BASE_URL}`);
check(accessPage, { 'highest first home 200': (resp) => resp.status === 200 });
sleep(1);
}
결과
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: smoke.js
output: -
scenarios: (100.00%) 1 scenario, 1 max VUs, 40s max duration (incl. graceful stop):
* default: 1 looping VUs for 10s (gracefulStop: 30s)
running (10.6s), 0/1 VUs, 10 complete and 0 interrupted iterations
default ✓ [======================================] 1 VUs 10s
✓ search paths 200
✓ add_station_paths_201
✓ highest first home 200
checks.........................: 100.00% ✓ 30 ✗ 0
data_received..................: 51 kB 4.8 kB/s
data_sent......................: 4.7 kB 447 B/s
http_req_blocked...............: avg=1.01ms min=4.06µs med=5.39µs max=30.26ms p(90)=14.96µs p(95)=27.13µs
http_req_connecting............: avg=12.75µs min=0s med=0s max=382.64µs p(90)=0s p(95)=0s
✓ http_req_duration..............: avg=19.24ms min=2.08ms med=8.24ms max=56.22ms p(90)=47.01ms p(95)=47.82ms
{ expected_response:true }...: avg=19.24ms min=2.08ms med=8.24ms max=56.22ms p(90)=47.01ms p(95)=47.82ms
http_req_failed................: 0.00% ✓ 0 ✗ 30
http_req_receiving.............: avg=73.4µs min=39.87µs med=65.14µs max=137.73µs p(90)=105.12µs p(95)=121.44µs
http_req_sending...............: avg=24.09µs min=12.06µs med=22.34µs max=74.73µs p(90)=30.69µs p(95)=47.19µs
http_req_tls_handshaking.......: avg=966.27µs min=0s med=0s max=28.98ms p(90)=0s p(95)=0s
http_req_waiting...............: avg=19.14ms min=2ms med=8.15ms max=56.09ms p(90)=46.87ms p(95)=47.68ms
http_reqs......................: 30 2.825944/s
iteration_duration.............: avg=1.06s min=1.05s med=1.05s max=1.08s p(90)=1.06s p(95)=1.07s
iterations.....................: 10 0.941981/s
vus............................: 1 min=1 max=1
Load TEST JS
-
너무 길어지는거 같아 여러 데이터를 참조하는 페이지만 올렸습니다.
-
일단 대략적인 평균 rps(30)를 주기 위해, 접속 빈도가 높은 페이지(20rps), 데이터를 갱신하는 페이지(10rps)를 주고 테스트를 하였습니다.
const BASE_URL = 'https://dongguri.kro.kr/';
export let options = {
scenarios: {
join_page_test: {
executor: 'ramping-vus',
startTime: '30s',
stages: [
{ duration: '10s', target: 100 }, // simulate ramp-up of traffic from 1 to 100 users over 5 minutes.
{ duration: '10s', target: 150 },
{ duration: '20s', target: 270 },
{ duration: '20s', target: 270 }, // Max Rps
{ duration: '10s', target: 150 }, // ramp-down to 0 users
{ duration: '10s', target: 0 }, // ramp-down to 0 users
],
exec: 'joinPageTest' // function scenario
},
main_page_test: {
executor: 'constant-vus',
vus: 20,
exec: 'mainPageTest', // function scenario
duration: '130s'
},
renew_page_test: {
executor: 'constant-vus',
vus: 10,
exec: 'renewPageTest', // function scenario
duration: '130s'
}
},
thresholds: {
http_req_duration: ['p(99)<1500'], // 99% of requests must complete below 1.5s
'http_req_duration{scenario:join_page_test}': ['p(99)<1500'], // 99% of requests must complete below 1.5s
'http_req_duration{scenario:main_page_test}': ['p(99)<1500'], // 99% of requests must complete below 1.5s
'http_req_duration{scenario:renew_page_test}': ['p(99)<1500'], // 99% of requests must complete below 1.5s
http_req_failed: ['rate<0.01'], // 0.1% Error
},
};
// 데이터를 조회하는데 여러 데이터를 참조하는 페이지:
export function joinPageTest() {
let searchPath = http.get(`${BASE_URL}/paths?source=2&target=1`);
check(searchPath, { 'search paths 200': (resp) => resp.status === 200 });
}
// 데이터를 갱신하는 페이지
export function renewPageTest() {
var randomName = Math.floor(Math.random() * 10000000000).toString();
let addStation = http.post(`${BASE_URL}/stations`, JSON.stringify({ name: randomName }), { headers: { 'Content-Type': 'application/json' } });
check(addStation,
{ 'add station paths 201': (resp) => resp.status === 201 });
}
// 접속 빈도가 높은 페이지
export function mainPageTest() {
let accessPage = http.get(`${BASE_URL}`);
check(accessPage, { 'highest first home 200': (resp) => resp.status === 200 });
}
결과
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: scenario-load.js
output: -
scenarios: (100.00%) 3 scenarios, 300 max VUs, 2m40s max duration (incl. graceful stop):
* main_page_test: 20 looping VUs for 2m10s (exec: mainPageTest, gracefulStop: 30s)
* renew_page_test: 10 looping VUs for 2m10s (exec: renewPageTest, gracefulStop: 30s)
* join_page_test: Up to 270 looping VUs for 1m20s over 6 stages (gracefulRampDown: 30s, exec: joinPageTest, startTime: 30s, gracefulStop: 30s)
running (2m10.0s), 000/300 VUs, 63192 complete and 0 interrupted iterations
main_page_test ✓ [======================================] 20 VUs 2m10s
renew_page_test ✓ [======================================] 10 VUs 2m10s
join_page_test ✓ [======================================] 000/270 VUs 1m20s
✓ highest first home 200
✓ add station paths 201
✓ search paths 200
checks...........................: 100.00% ✓ 63192 ✗ 0
data_received....................: 86 MB 659 kB/s
data_sent........................: 8.7 MB 67 kB/s
http_req_blocked.................: avg=157.15µs min=2.93µs med=4.13µs max=125.68ms p(90)=4.93µs p(95)=9.41µs
http_req_connecting..............: avg=9.12µs min=0s med=0s max=43.89ms p(90)=0s p(95)=0s
✗ http_req_duration................: avg=290.21ms min=1.84ms med=41.31ms max=7.46s p(90)=1.67s p(95)=1.81s
{ expected_response:true }.....: avg=290.21ms min=1.84ms med=41.31ms max=7.46s p(90)=1.67s p(95)=1.81s
✗ { scenario:join_page_test }....: avg=1.92s min=131.14ms med=1.79s max=7.46s p(90)=2.81s p(95)=3.27s
✓ { scenario:main_page_test }....: avg=62.86ms min=1.84ms med=39.11ms max=654.16ms p(90)=106.26ms p(95)=190.86ms
✗ { scenario:renew_page_test }...: avg=89.52ms min=7.47ms med=34.45ms max=6.69s p(90)=65.61ms p(95)=93.85ms
✓ http_req_failed..................: 0.00% ✓ 0 ✗ 63192
http_req_receiving...............: avg=53.64µs min=22.95µs med=42.96µs max=27.16ms p(90)=68.42µs p(95)=79.26µs
http_req_sending.................: avg=20.87µs min=8.18µs med=12.85µs max=19.51ms p(90)=20.79µs p(95)=38.29µs
http_req_tls_handshaking.........: avg=140µs min=0s med=0s max=124.96ms p(90)=0s p(95)=0s
http_req_waiting.................: avg=290.14ms min=1.75ms med=41.23ms max=7.46s p(90)=1.67s p(95)=1.81s
http_reqs........................: 63192 485.963539/s
iteration_duration...............: avg=290.5ms min=1.95ms med=41.51ms max=7.46s p(90)=1.67s p(95)=1.81s
iterations.......................: 63192 485.963539/s
vus..............................: 30 min=30 max=300
vus_max..........................: 300 min=300 max=300
Stress Test js
- Load 테스트와 조건을 같은 조건을 주었습니다
import http from 'k6/http';
import { check, group, sleep, fail } from 'k6';
const BASE_URL = 'https://dongguri.kro.kr/';
export let options = {
scenarios: {
join_page_test: {
executor: 'ramping-vus',
startTime: '20s',
stages: [
{ duration: '10s', target: 100 }, // simulate ramp-up of traffic from 1 to 100 users over 5 minutes.
{ duration: '10s', target: 100 },
{ duration: '10s', target: 200 },
{ duration: '10s', target: 200 }, // ramp-up
{ duration: '10s', target: 270 },
{ duration: '10s', target: 270 }, // 예측rps max 지점
{ duration: '10s', target: 500 }, // ramp-up
{ duration: '10s', target: 500 },
{ duration: '10s', target: 600 }, // ramp-up
{ duration: '10s', target: 600 },
{ duration: '10s', target: 700 }, // ramp-up
{ duration: '10s', target: 700 },
{ duration: '10s', target: 0 },
],
exec: 'joinPageTest' // function scenario
},
main_page_test: {
executor: 'constant-vus',
vus: 20,
exec: 'mainPageTest', // function scenario
duration: '130s'
},
renew_page_test: {
executor: 'constant-vus',
vus: 10,
exec: 'renewPageTest', // function scenario
duration: '130s'
}
},
thresholds: {
http_req_duration: ['p(99)<1500'], // 99% of requests must complete below 1.5s
'http_req_duration{scenario:join_page_test}': ['p(99)<1500'], // 99% of requests must complete below 1.5s
'http_req_duration{scenario:main_page_test}': ['p(99)<1500'], // 99% of requests must complete below 1.5s
'http_req_duration{scenario:renew_page_test}': ['p(99)<1500'], // 99% of requests must complete below 1.5s
http_req_failed: ['rate<0.01'], // 0.1% Error
},
};
// 데이터를 조회하는데 여러 데이터를 참조하는 페이지:
export function joinPageTest() {
let searchPath = http.get(`${BASE_URL}/paths?source=2&target=1`);
check(searchPath, { 'search paths 200': (resp) => resp.status === 200 });
}
// 데이터를 갱신하는 페이지
export function renewPageTest() {
var randomName = Math.floor(Math.random() * 10000000000).toString();
let addStation = http.post(`${BASE_URL}/stations`, JSON.stringify({ name: randomName }), { headers: { 'Content-Type': 'application/json' } });
check(addStation,
{ 'add station paths 201': (resp) => resp.status === 201 });
}
// 접속 빈도가 높은 페이지
export function mainPageTest() {
let accessPage = http.get(`${BASE_URL}`);
check(accessPage, { 'highest first home 200': (resp) => resp.status === 200 });
}
결과
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: scenario-stress.js
output: -
scenarios: (100.00%) 3 scenarios, 730 max VUs, 3m0s max duration (incl. graceful stop):
* main_page_test: 20 looping VUs for 2m10s (exec: mainPageTest, gracefulStop: 30s)
* renew_page_test: 10 looping VUs for 2m10s (exec: renewPageTest, gracefulStop: 30s)
* join_page_test: Up to 700 looping VUs for 2m10s over 13 stages (gracefulRampDown: 30s, exec: joinPageTest, startTime: 20s, gracefulStop: 30s)
running (2m30.0s), 000/730 VUs, 45933 complete and 0 interrupted iterations
main_page_test ✓ [======================================] 20 VUs 2m10s
renew_page_test ✓ [======================================] 10 VUs 2m10s
join_page_test ✓ [======================================] 000/700 VUs 2m10s
✓ highest first home 200
✓ add station paths 201
✓ search paths 200
checks...........................: 100.00% ✓ 45933 ✗ 0
data_received....................: 85 MB 567 kB/s
data_sent........................: 6.4 MB 42 kB/s
http_req_blocked.................: avg=215.79µs min=2.97µs med=4.13µs max=97.08ms p(90)=5.21µs p(95)=11.72µs
http_req_connecting..............: avg=21.62µs min=0s med=0s max=27.3ms p(90)=0s p(95)=0s
✗ http_req_duration................: avg=1.14s min=1.78ms med=70.55ms max=9.73s p(90)=3.94s p(95)=4.35s
{ expected_response:true }.....: avg=1.14s min=1.78ms med=70.55ms max=9.73s p(90)=3.94s p(95)=4.35s
✗ { scenario:join_page_test }....: avg=3.09s min=59.08ms med=3.21s max=9.73s p(90)=4.4s p(95)=4.43s
✗ { scenario:main_page_test }....: avg=110.73ms min=1.78ms med=41.65ms max=3.18s p(90)=182.8ms p(95)=310.9ms
✗ { scenario:renew_page_test }...: avg=203.52ms min=7.55ms med=32.92ms max=7.75s p(90)=92.48ms p(95)=1.71s
✓ http_req_failed..................: 0.00% ✓ 0 ✗ 45933
http_req_receiving...............: avg=54.82µs min=22.77µs med=46.47µs max=14.77ms p(90)=72.86µs p(95)=85.75µs
http_req_sending.................: avg=22.54µs min=8.45µs med=13.02µs max=24.16ms p(90)=22.4µs p(95)=41.08µs
http_req_tls_handshaking.........: avg=182.33µs min=0s med=0s max=95.88ms p(90)=0s p(95)=0s
http_req_waiting.................: avg=1.14s min=1.72ms med=70.48ms max=9.73s p(90)=3.94s p(95)=4.35s
http_reqs........................: 45933 306.132856/s
iteration_duration...............: avg=1.14s min=1.87ms med=70.95ms max=9.73s p(90)=3.94s p(95)=4.35s
iterations.......................: 45933 306.132856/s
vus..............................: 10 min=10 max=729
vus_max..........................: 730 min=730 max=730
ERRO[0152] some thresholds have failed
감사합니다~~
from infra-subway.
oeeen
commented on August 17, 2024
1
- 잘 알지는 못하지만, 일반적인 웹 성능 예상 지표를 기준으로 삼아서 선정할 수도 있을 것 같습니다. PageSpeed 점수 몇점 이상 이라는 조건을 걸거나, 강의 자료에서 나온 수치들로 설정한다거나 할 수 있을 것 같습니다. 서비스의 타겟과 시간대, 응답시간이 실시간 급으로 중요한 서비스라면 좀 더 성능지표가 중요할 것이고, 실시간성으로 응답할 필요까진 없는 서비스의 경우에는 앞의 수치보다는 낮아도 될 것이라고 생각합니다. 물론 자원이 풍부하고, 예산이 넉넉하고 막 써도 되는 수준이라면 성능은 무조건적으로 향상시키면 좋을 것이고 더 좋은 CPU, 메모리 등의 스펙을 증가시켜 응답속도를 올리고 정적파일 캐싱등으로 최대한 끌어올릴 수 있을 것으로 생각합니다만, 자원은 한정되어 있고 성능에 투자할 예산과 그에 대한 리턴에 대한 비교를 해서 적절한 수치를 정해야할 것이라고 생각합니다.
- WebPageTest 에서 Largest Contentful Paint 부분이 시간이 가장 오래 걸리고, PageSpeed에서는 개선을 위해 텍스트 압축 사용과 사용하지 않는 자바스크립트 파일 제거를 추천해서, 이 부분을 개선하는 것이 좋을 것 같습니다. LCP는 서버 최적화, 사용자와 가까운 CDN 사용, 캐싱 등을 활용하여 최적화 할 수 있다고 하는데, 이 실습과제에서는 빠르게 해볼 수 있는 부분은 없는 것 같아서 시도하지 않았습니다. (참고 - https://ui.toast.com/weekly-pick/ko_202012101720) 간단하게 적용할 수 있는 방법으로 컨텐츠 압축만 해줬습니다. (https://docs.spring.io/spring-boot/docs/2.4.0/reference/html/howto.html#howto-embedded-web-servers)
- 부하테스트 전제조건
1. 예상 1일 사용자 수: 300,000
2. 피크시간: 출,퇴근 시간 집중
3. 1명당 1일 평균 접속: 5번
4. Throughput
- 1일 총 접속 수: 1,500,000
- 1일 평균 rps: 17.3 => 18로 생각
- 1일 최대 rps: 18 * 10 = 180
5. 시나리오 대상: 메인페이지, 경로검색
6. 응답속도: 피크타임에 500ms 이하
Smoke test (VUser 1, duration 10s)
메인 페이지
data_received..................: 16 kB 1.5 kB/s
data_sent......................: 1.5 kB 151 B/s
http_req_blocked...............: avg=8.54ms min=20.6µs med=29.25µs max=85.1ms p(90)=8.59ms p(95)=46.85ms
http_req_connecting............: avg=596.42µs min=0s med=0s max=5.96ms p(90)=596.41µs p(95)=3.28ms
✓ http_req_duration..............: avg=8.14ms min=6.56ms med=7.38ms max=15.43ms p(90)=8.83ms p(95)=12.13ms
{ expected_response:true }...: avg=8.14ms min=6.56ms med=7.38ms max=15.43ms p(90)=8.83ms p(95)=12.13ms
http_req_failed................: 0.00% ✓ 0 ✗ 10
http_req_receiving.............: avg=184.03µs min=91.7µs med=167.3µs max=377.7µs p(90)=265.91µs p(95)=321.8µs
http_req_sending...............: avg=93.11µs min=34.8µs med=51.55µs max=363µs p(90)=147.89µs p(95)=255.44µs
http_req_tls_handshaking.......: avg=6.15ms min=0s med=0s max=61.58ms p(90)=6.15ms p(95)=33.87ms
http_req_waiting...............: avg=7.87ms min=6.03ms med=7.19ms max=15.16ms p(90)=8.54ms p(95)=11.85ms
http_reqs......................: 10 0.981457/s
iteration_duration.............: avg=1.01s min=1s med=1s max=1.09s p(90)=1.02s p(95)=1.06s
iterations.....................: 10 0.981457/s
vus............................: 1 min=1 max=1
vus_max........................: 1 min=1 max=1
경로검색 (1->2)
data_received..................: 7.7 kB 741 B/s
data_sent......................: 1.8 kB 170 B/s
http_req_blocked...............: avg=9.44ms min=31.1µs med=43.45µs max=93.89ms p(90)=9.54ms p(95)=51.71ms
http_req_connecting............: avg=652.17µs min=0s med=0s max=6.52ms p(90)=652.16µs p(95)=3.58ms
✓ http_req_duration..............: avg=30.87ms min=25.54ms med=27.53ms max=51.34ms p(90)=39.06ms p(95)=45.2ms
{ expected_response:true }...: avg=30.87ms min=25.54ms med=27.53ms max=51.34ms p(90)=39.06ms p(95)=45.2ms
http_req_failed................: 0.00% ✓ 0 ✗ 10
http_req_receiving.............: avg=293.69µs min=111.4µs med=217.2µs max=926.6µs p(90)=491.26µs p(95)=708.93µs
http_req_sending...............: avg=133.18µs min=42.9µs med=99.95µs max=338.1µs p(90)=257.99µs p(95)=298.04µs
http_req_tls_handshaking.......: avg=8.29ms min=0s med=0s max=82.97ms p(90)=8.29ms p(95)=45.63ms
http_req_waiting...............: avg=30.44ms min=25.26ms med=27.22ms max=50.17ms p(90)=38.81ms p(95)=44.49ms
http_reqs......................: 10 0.962154/s
iteration_duration.............: avg=1.04s min=1.02s med=1.02s max=1.14s p(90)=1.05s p(95)=1.09s
iterations.....................: 10 0.962154/s
vus............................: 1 min=1 max=1
vus_max........................: 1 min=1 max=1
Load Test
- 평균 rps 20 target으로 돌다가 최대 rps 예측치인 180 (스크립트에선 200으로 설정) -> 다시 평균 rps로 돌아가고 0으로 돌아가는 시나리오
메인 페이지
data_received..................: 22 MB 166 kB/s
data_sent......................: 2.0 MB 16 kB/s
http_req_blocked...............: avg=284.45µs min=18.7µs med=29.2µs max=87.25ms p(90)=69.9µs p(95)=114.9µs
http_req_connecting............: avg=71.84µs min=0s med=0s max=28.04ms p(90)=0s p(95)=0s
✓ http_req_duration..............: avg=6.71ms min=3.84ms med=6ms max=218.65ms p(90)=7.44ms p(95)=8.53ms
{ expected_response:true }...: avg=6.71ms min=3.84ms med=6ms max=218.65ms p(90)=7.44ms p(95)=8.53ms
http_req_failed................: 0.00% ✓ 0 ✗ 16661
http_req_receiving.............: avg=167.88µs min=55.7µs med=140.2µs max=3.64ms p(90)=269µs p(95)=333.2µs
http_req_sending...............: avg=75.68µs min=19.2µs med=46.5µs max=6.53ms p(90)=136.8µs p(95)=203.1µs
http_req_tls_handshaking.......: avg=168.18µs min=0s med=0s max=58.97ms p(90)=0s p(95)=0s
http_req_waiting...............: avg=6.47ms min=3.69ms med=5.76ms max=218.54ms p(90)=7.17ms p(95)=8.18ms
http_reqs......................: 16661 128.236824/s
iteration_duration.............: avg=1s min=1s med=1s max=1.31s p(90)=1s p(95)=1.01s
iterations.....................: 16661 128.236824/s
vus............................: 2 min=1 max=200
vus_max........................: 200 min=200 max=200
경로검색 (1->2)
data_received..................: 8.4 MB 64 kB/s
data_sent......................: 2.3 MB 18 kB/s
http_req_blocked...............: avg=295.74µs min=16µs med=30.9µs max=71.07ms p(90)=70.56µs p(95)=119.2µs
http_req_connecting............: avg=75.74µs min=0s med=0s max=30.37ms p(90)=0s p(95)=0s
✓ http_req_duration..............: avg=30.29ms min=151.4µs med=25.76ms max=273.89ms p(90)=43.91ms p(95)=55.65ms
{ expected_response:true }...: avg=30.29ms min=151.4µs med=25.76ms max=273.89ms p(90)=43.91ms p(95)=55.65ms
http_req_failed................: 0.00% ✓ 0 ✗ 16285
http_req_receiving.............: avg=169.37µs min=55.4µs med=141.3µs max=3.33ms p(90)=272.6µs p(95)=337.91µs
http_req_sending...............: avg=71.62µs min=18.6µs med=48.1µs max=3.01ms p(90)=133.16µs p(95)=184.4µs
http_req_tls_handshaking.......: avg=174.09µs min=0s med=0s max=52.81ms p(90)=0s p(95)=0s
http_req_waiting...............: avg=30.05ms min=0s med=25.52ms max=273.65ms p(90)=43.7ms p(95)=55.46ms
http_reqs......................: 16285 124.554669/s
iteration_duration.............: avg=1.03s min=1.01s med=1.02s max=1.27s p(90)=1.04s p(95)=1.05s
iterations.....................: 16285 124.554669/s
vus............................: 2 min=1 max=200
vus_max........................: 200 min=200 max=200
Stress test
export let options = {
thresholds: {
http_req_duration: ['p(99)<500'], // 99% of requests must complete below 0.5s
},
stages: [
{ duration: '20s', target: 100 },
{ duration: '20s', target: 200 },
{ duration: '20s', target: 400 },
{ duration: '20s', target: 600 },
{ duration: '20s', target: 800 },
{ duration: '20s', target: 1000 },
{ duration: '10s', target: 200 },
{ duration: '10s', target: 1000 },
{ duration: '10s', target: 20 },
{ duration: '10s', target: 0 },
],
};메인 페이지
data_received..................: 106 MB 621 kB/s
data_sent......................: 12 MB 72 kB/s
http_req_blocked...............: avg=65.79ms min=0s med=51.9µs max=523.31ms p(90)=252.7ms p(95)=296.8ms
http_req_connecting............: avg=9.51ms min=-14.4063ms med=0s max=222.26ms p(90)=30.3ms p(95)=54.67ms
✓ http_req_duration..............: avg=33.28ms min=-28.301ms med=7ms max=29.97s p(90)=48.78ms p(95)=85.1ms
{ expected_response:true }...: avg=20.55ms min=-28.301ms med=7.21ms max=559.39ms p(90)=50.29ms p(95)=87.26ms
http_req_failed................: 3.40% ✓ 1443 ✗ 40964
http_req_receiving.............: avg=173.87µs min=-33.8895ms med=116.1µs max=197.6ms p(90)=258.29µs p(95)=357.96µs
http_req_sending...............: avg=2ms min=0s med=86.3µs max=460.34ms p(90)=907.62µs p(95)=1.95ms
http_req_tls_handshaking.......: avg=54.57ms min=-22.1182ms med=0s max=510.43ms p(90)=212.51ms p(95)=255.2ms
http_req_waiting...............: avg=31.09ms min=0s med=6.72ms max=29.97s p(90)=43.66ms p(95)=72.02ms
http_reqs......................: 42407 248.122095/s
iteration_duration.............: avg=2.09s min=1s med=1.01s max=31.11s p(90)=1.33s p(95)=1.44s
iterations.....................: 42394 248.046032/s
vus............................: 4 min=4 max=1000
vus_max........................: 1000 min=1000 max=1000
경로검색 (1->2)
export let options = {
thresholds: {
http_req_duration: ['p(99)<500'], // 99% of requests must complete below 0.5s
},
stages: [
{ duration: '20s', target: 100 },
{ duration: '20s', target: 200 },
{ duration: '20s', target: 400 },
{ duration: '20s', target: 600 },
{ duration: '20s', target: 800 },
{ duration: '10s', target: 200 },
{ duration: '10s', target: 0 },
],
};결과
data_received..................: 62 MB 516 kB/s
data_sent......................: 11 MB 92 kB/s
http_req_blocked...............: avg=56.82ms min=0s med=18.54ms max=2.26s p(90)=119.72ms p(95)=332.98ms
http_req_connecting............: avg=13.6ms min=-28.3272ms med=4.68ms max=396.42ms p(90)=26.3ms p(95)=82.34ms
✗ http_req_duration..............: avg=313.15ms min=-24.2849ms med=43.49ms max=2.97s p(90)=783.63ms p(95)=883.47ms
{ expected_response:true }...: avg=348.68ms min=120µs med=142.06ms max=2.97s p(90)=794.63ms p(95)=922.55ms
http_req_failed................: 10.74% ✓ 3174 ✗ 26357
http_req_receiving.............: avg=219.69µs min=-33.4132ms med=126.2µs max=110.79ms p(90)=292.9µs p(95)=422.5µs
http_req_sending...............: avg=834.14µs min=0s med=105µs max=589.43ms p(90)=836.5µs p(95)=2.08ms
http_req_tls_handshaking.......: avg=42.87ms min=-18.4099ms med=13.08ms max=2.15s p(90)=88.67ms p(95)=221.57ms
http_req_waiting...............: avg=312.1ms min=-24.4045ms med=42.83ms max=2.97s p(90)=782.67ms p(95)=879.76ms
http_reqs......................: 29531 244.93384/s
iteration_duration.............: avg=1.37s min=1s med=1.07s max=3.99s p(90)=1.88s p(95)=2.15s
iterations.....................: 29531 244.93384/s
vus............................: 12 min=5 max=799
vus_max........................: 800 min=800 max=800
- Stress Test는 어느정도가 max 일지 예측이 잘 안돼서, 무작정 계속 vuser를 올려봤습니다.
- 개인 맥북 docker container위에서 EC2 service에 테스트를 진행했습니다.
- nGrinder를 사용해야 할 일이 있어서 이번 미션에서도 nGrinder로 진행하려고 했으나,, 일단은 k6로 진행했습니다.
- 테스트 자체가 잘못된건지 평균응답 속도가 굉장히 빠른 것 같네요
감사합니다!
from infra-subway.
wooyongha
commented on August 17, 2024
1
-
웹 성능예산은 어느정도가 적당하다고 생각하시나요
-
일반적인 사용자 서비스 / 이미지가 많은 화면 / 데이터가 많은 화면 등 서비스마다 기준을 다르게 가져갈 수 있을 것 같습니다.
-
지하철 노선도 서비스의 경우 1.5s 내로 떨어져야 사용자 편의성을 해치지 않을것으로 예상하여 테스트를 진행하였습니다.
-
-
웹 성능예산을 바탕으로 현재 지하철 노선도 서비스는 어떤 부분을 개선하면 좋을까요
- 미사용 js 파일 삭제
- 콘텐츠 압축
- 사용자 인증을 체크하는 부분은 캐시 서버를 활용해도 좋을 것 같습니다.
-
부하테스트 전제조건은 어느정도로 설정하셨나요
- 요청은 1.5 s 내로 응답해야한다.
- 피크 시간의 경우 평소의 10배의 트래픽으로 가정
- 목푯값
- 예상 1일 DAU : 20만
- 1명당 1일 평균 요청 수 : 10
- 1일 총 접속수 : 200만
- 1일 평균 rps : 23 rps
- 1일 최대 rps : 230 rps
-
Smoke, Load, Stress 테스트 결과를 공유해주세요
사용자가 가장 많은 요청을 하는 메인페이지와 가장 많은 리소스를 사용하는 경로 검색 서비스를 기준으로 테스트 진행하였습니다.
Smoke Test
VUser = 1, duration = 10s
메인 페이지
checks.........................: 100.00% ✓ 1479 ✗ 0 data_received..................: 1.9 MB 191 kB/s data_sent......................: 167 kB 17 kB/s http_req_blocked...............: avg=277.43µs min=1µs med=4µs max=185.28ms p(90)=6µs p(95)=6µs http_req_connecting............: avg=48.43µs min=0s med=0s max=7.06ms p(90)=0s p(95)=0s ✓ http_req_duration..............: avg=6.3ms min=3.83ms med=5.36ms max=94.56ms p(90)=6.93ms p(95)=8.11ms { expected_response:true }...: avg=6.3ms min=3.83ms med=5.36ms max=94.56ms p(90)=6.93ms p(95)=8.11ms http_req_failed................: 0.00% ✓ 0 ✗ 1479 http_req_receiving.............: avg=74.24µs min=27µs med=74µs max=203µs p(90)=100µs p(95)=109µs http_req_sending...............: avg=25.47µs min=8µs med=23µs max=1.14ms p(90)=32µs p(95)=34.09µs http_req_tls_handshaking.......: avg=218.91µs min=0s med=0s max=173.95ms p(90)=0s p(95)=0s http_req_waiting...............: avg=6.2ms min=3.76ms med=5.25ms max=94.44ms p(90)=6.82ms p(95)=7.99ms http_reqs......................: 1479 147.705516/s iteration_duration.............: avg=6.75ms min=3.96ms med=5.54ms max=197.01ms p(90)=7.23ms p(95)=8.95ms iterations.....................: 1479 147.705516/s vus............................: 1 min=1 max=1 vus_max........................: 1 min=1 max=1경로 검색
checks.........................: 100.00% ✓ 10 ✗ 0 data_received..................: 16 kB 1.5 kB/s data_sent......................: 1.7 kB 159 B/s http_req_blocked...............: avg=16.11ms min=6µs med=6.5µs max=161.04ms p(90)=16.11ms p(95)=88.57ms http_req_connecting............: avg=485.1µs min=0s med=0s max=4.85ms p(90)=485.09µs p(95)=2.66ms ✓ http_req_duration..............: avg=37.19ms min=6.72ms med=46.21ms max=51.35ms p(90)=51.2ms p(95)=51.27ms { expected_response:true }...: avg=37.19ms min=6.72ms med=46.21ms max=51.35ms p(90)=51.2ms p(95)=51.27ms http_req_failed................: 0.00% ✓ 0 ✗ 10 http_req_receiving.............: avg=93.5µs min=50µs med=87µs max=181µs p(90)=108.99µs p(95)=144.99µs http_req_sending...............: avg=159.7µs min=26µs med=36µs max=1.28ms p(90)=164.69µs p(95)=725.84µs http_req_tls_handshaking.......: avg=15.51ms min=0s med=0s max=155.17ms p(90)=15.51ms p(95)=85.34ms http_req_waiting...............: avg=36.94ms min=6.58ms med=46.11ms max=51.24ms p(90)=51.07ms p(95)=51.16ms http_reqs......................: 10 0.947602/s iteration_duration.............: avg=1.05s min=1.01s med=1.04s max=1.17s p(90)=1.06s p(95)=1.12s iterations.....................: 10 0.947602/s vus............................: 1 min=1 max=1 vus_max........................: 1 min=1 max=1Load Test
export let options = { stages: [ { duration: '10s', target: 30 }, { duration: '20s', target: 100 }, { duration: '30s', target: 250 }, { duration: '30s', target: 250 }, { duration: '20s', target: 100 }, { duration: '10s', target: 30 }, { duration: '10s', target: 0 } ], thresholds: { http_req_duration: ['p(99)<1500'], // 99% of requests must complete below 1.5s }, };
메인 페이지
checks.........................: 100.00% ✓ 17792 ✗ 0 data_received..................: 23 MB 178 kB/s data_sent......................: 2.1 MB 16 kB/s http_req_blocked...............: avg=438.08µs min=1µs med=4µs max=244.64ms p(90)=7µs p(95)=8µs http_req_connecting............: avg=169.71µs min=0s med=0s max=85.4ms p(90)=0s p(95)=0s ✓ http_req_duration..............: avg=45.73ms min=3.95ms med=42.01ms max=1.25s p(90)=64.9ms p(95)=74.42ms { expected_response:true }...: avg=45.73ms min=3.95ms med=42.01ms max=1.25s p(90)=64.9ms p(95)=74.42ms http_req_failed................: 0.00% ✓ 0 ✗ 17792 http_req_receiving.............: avg=54.54µs min=14µs med=46µs max=1.17ms p(90)=90µs p(95)=105µs http_req_sending...............: avg=23.09µs min=5µs med=17µs max=1ms p(90)=35µs p(95)=49µs http_req_tls_handshaking.......: avg=261.9µs min=0s med=0s max=225.24ms p(90)=0s p(95)=0s http_req_waiting...............: avg=45.65ms min=3.86ms med=41.95ms max=1.25s p(90)=64.84ms p(95)=74.35ms http_reqs......................: 17792 136.262154/s iteration_duration.............: avg=1.04s min=1s med=1.04s max=2.25s p(90)=1.06s p(95)=1.07s iterations.....................: 17792 136.262154/s vus............................: 2 min=2 max=250 vus_max........................: 250 min=250 max=250경로 검색
checks.........................: 100.00% ✓ 18220 ✗ 0 data_received..................: 24 MB 182 kB/s data_sent......................: 2.5 MB 19 kB/s http_req_blocked...............: avg=371.28µs min=1µs med=4µs max=164.09ms p(90)=7µs p(95)=8µs http_req_connecting............: avg=135.07µs min=0s med=0s max=88.67ms p(90)=0s p(95)=0s ✓ http_req_duration..............: avg=21.06ms min=3.61ms med=12.16ms max=1.05s p(90)=49.59ms p(95)=56.52ms { expected_response:true }...: avg=21.06ms min=3.61ms med=12.16ms max=1.05s p(90)=49.59ms p(95)=56.52ms http_req_failed................: 0.00% ✓ 0 ✗ 18220 http_req_receiving.............: avg=47.69µs min=15µs med=38µs max=1.03ms p(90)=84µs p(95)=96µs http_req_sending...............: avg=20.33µs min=5µs med=15µs max=985µs p(90)=34µs p(95)=43µs http_req_tls_handshaking.......: avg=230.45µs min=0s med=0s max=155.42ms p(90)=0s p(95)=0s http_req_waiting...............: avg=20.99ms min=3.56ms med=12.1ms max=1.05s p(90)=49.51ms p(95)=56.47ms http_reqs......................: 18220 139.165485/s iteration_duration.............: avg=1.02s min=1s med=1.01s max=2.05s p(90)=1.05s p(95)=1.05s iterations.....................: 18220 139.165485/s vus............................: 1 min=1 max=250 vus_max........................: 250 min=250 max=250Stress Test
export let options = { stages: [ { duration: '10s', target: 30 }, { duration: '10s', target: 100 }, { duration: '10s', target: 300 }, { duration: '10s', target: 500 }, { duration: '10s', target: 700 }, { duration: '10s', target: 800 }, { duration: '10s', target: 400 }, { duration: '10s', target: 0 } ], thresholds: { http_req_duration: ['p(99)<1500'], // 99% of requests must complete below 1.5s }, }
메인 페이지
checks.........................: 99.07% ✓ 87367 ✗ 813 data_received..................: 118 MB 1.1 MB/s data_sent......................: 11 MB 99 kB/s http_req_blocked...............: avg=130.13ms min=0s med=2µs max=26.81s p(90)=5µs p(95)=6µs http_req_connecting............: avg=851.61µs min=0s med=0s max=7.12s p(90)=0s p(95)=0s ✓ http_req_duration..............: avg=185.26ms min=0s med=70.59ms max=28.02s p(90)=479.42ms p(95)=496.43ms { expected_response:true }...: avg=174.77ms min=3.44ms med=70.83ms max=28.02s p(90)=478.64ms p(95)=494.87ms http_req_failed................: 0.92% ✓ 813 ✗ 87367 http_req_receiving.............: avg=43.14µs min=0s med=36µs max=758µs p(90)=72µs p(95)=88µs http_req_sending...............: avg=16.26ms min=0s med=12µs max=28.02s p(90)=25µs p(95)=32µs http_req_tls_handshaking.......: avg=113.48ms min=0s med=0s max=26.8s p(90)=0s p(95)=0s http_req_waiting...............: avg=168.95ms min=0s med=69.06ms max=19.15s p(90)=477.04ms p(95)=492.67ms http_reqs......................: 88180 814.331037/s iteration_duration.............: avg=328ms min=3.52ms med=72.83ms max=28.02s p(90)=484.12ms p(95)=506.76ms iterations.....................: 88180 814.331037/s vus............................: 1 min=1 max=800 vus_max........................: 800 min=800 max=800경로 검색 (vus=1000)
checks.........................: 99.63% ✓ 26085 ✗ 95 data_received..................: 87 MB 1.1 MB/s data_sent......................: 11 MB 134 kB/s http_req_blocked...............: avg=195.65ms min=1µs med=87.64ms max=6.86s p(90)=497.38ms p(95)=594.66ms http_req_connecting............: avg=4.62ms min=0s med=4.48ms max=172.82ms p(90)=7.17ms p(95)=8.52ms ✗ http_req_duration..............: avg=52.78ms min=0s med=11.57ms max=36.03s p(90)=16.43ms p(95)=20.88ms { expected_response:true }...: avg=47ms min=3.4ms med=11.57ms max=4.97s p(90)=16.38ms p(95)=20.42ms http_req_failed................: 0.36% ✓ 95 ✗ 26085 http_req_receiving.............: avg=44.57µs min=0s med=35µs max=1.54ms p(90)=77µs p(95)=93µs http_req_sending...............: avg=31.59ms min=0s med=30µs max=4.95s p(90)=58µs p(95)=93µs http_req_tls_handshaking.......: avg=159.71ms min=0s med=51.65ms max=6.86s p(90)=455.5ms p(95)=581.16ms http_req_waiting...............: avg=21.14ms min=0s med=11.44ms max=36.03s p(90)=16.06ms p(95)=18.74ms http_reqs......................: 26180 322.845217/s iteration_duration.............: avg=1.21s min=1s med=1.1s max=38.01s p(90)=1.52s p(95)=1.61s iterations.....................: 26180 322.845217/s vus............................: 1 min=1 max=1000 vus_max........................: 1000 min=1000 max=1000해당 조건으로 부하를 지속할 경우 1.5s 이상 소요되는 요청 발생
from infra-subway.
brainbackdoor
commented on August 17, 2024
@mook1594
넵~ 잘 진행해주셨어요~ Stress Test를 진행할 때는 어느 지점부터 latency 가 급격히 증가하는지 확인해보시는게 좋을거 같아요~
그리고 어떤 형태로, 어떤 부분에서 문제가 발생하는지도 확인해보면 좋을거 같아요.
좋은 하루 보내세요!
from infra-subway.
vsh123
commented on August 17, 2024
- 지하철 노선도의 경우 대체로 비로그인 유저도 사용할 수 있는 화면이 많은 걸 감안하여
- 비로그인 화면 : 1s 내외
- 로그인 하여 사용할 수 있는 화면 1.5s내외
로 설정하면 좋을 것 같습니다!
- PageSpeed에서 가장 크게 제안했던 압축부분을 적용해보았습니다.
- 쉽게 변하지 않는 지하철 노선의 특성상 경로 검색 부분에서는 캐싱을 적용한다면 훨씬 빠른 응답을 기대할 것으로 보입니다.
3,4번 및 2번에서 Gzip으로 압축을 적용한 결과는 notion링크로 공유드립니다!
https://www.notion.so/mrvan/fc9687d889384b3eb1a71929c96b74ac
from infra-subway.
mindock
commented on August 17, 2024
웹 성능 예산
부하 테스트
from infra-subway.
Related Issues (8)
- 1. 서비스 구성하기 HOT 16
- 7. 쿠버네티스로 구성하기 HOT 1
- 8. 클러스터 운영하기 HOT 1
- 2. 서비스 배포하기 HOT 11
- 3. 서비스 운영하기 HOT 9
- 5. 화면 응답 개선하기 HOT 4
- 6. 조회 성능 개선하기 HOT 9
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from infra-subway.