Code Monkey home page Code Monkey logo

arcus-misc's People

Contributors

computerphilosopher avatar hjyun328 avatar jhpark816 avatar minwoojin avatar whchoi83 avatar yskimcau avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

jooho812 computerphilosopher suhwanjang

arcus-misc's Issues

[memc_mon] memcached 정상 shutdown 후 바로 재기동시 오동작 현상

Moved from
arcus-task/issue#36
arcus-task/issue#53

arcus-task 리포지토리에 여러 개의 memc_mon 관련 이슈에 코멘트가 복잡하게 얽혀서 이쪽으로 옮겨 재정리합니다.

최초 이슈

증상

file delete detection 후 process 존재 여부 check를 위해 kill() 호출 사이에 sleep time으로도 사용되는
monitoring 주기를 짧게(예... 50ms) 설정하여 실행한 경우 드물지만 아래와 같은 문제가 발생합니다.

  • pid file delete detection 후 kill() 호출 시 아직 memcached process가 살아 있는 경우가 존재하여 unregister 안 하게 되는 경우가 발생
  • 이후 동일한 port의 memcached를 재기동하면 동일 주소로 2개의 memcached 정보 등록.
  • 이후 완전히 shutdown된 process의 정보를 제거할 때... 새로 기동된 memcached 가 생성한 znode 정보를 삭제해 버려... 새로 기동된 memcached가 제대로 기동되지 않으며 master/slave 구성이 되지 않는 현상이 존재합니다.

수정 커밋

commit : jam2in/arcus-misc@e5f69ab

2번째 이슈

증상

시험 방법

  • 1개의 노드로만 시험 (master)
  • 다음을 반복 수행
    • ./kill.memcached.bash master 11215 INT
    • ./killandrun.memcached.bash master 11215 NONE

memcached 로그와 arcus_memc_mon 로그 확인하여
memcached 정상 기동 여부와 znode 정상 삭제 여부 확인

위 시험 방법으로 시험하면 pid file이 삭제되는 현상이 있습니다.

원인은 다음과 같습니다.

  • 이전 memcached shutdown 진행
    • 아직 이전 memcached process가 완전히 shutdown하기 전에 새로운 memcached 기동
    • 새로운 memcached 기동 완료
  • 이전 memcached의 shutdown 남은 작업 진행 (pid file 삭제 포함)

이슈 #53 반영하여 시험하였을 때의 monitoring process 로그입니다.

=== Checking memcached process (1/1) - pid : 13972 - addr : 127.0.0.1:11215 ... OK
=== Checking memcached process (1/1) - pid : 13972 - addr : 127.0.0.1:11215 ... OK
=== Detection : pid file deleted : memcached.127.0.0.1:11215
Memcached process exists. Do not unregister. : (-1, 127.0.0.1:11215)
Pid file deleted, but cannot unregister 127.0.0.1:11215 memcached node!
=== Checking memcached process (1/1) - pid : 13972 - addr : 127.0.0.1:11215 ... OK
=== Detection : pid file created : memcached.127.0.0.1:11215
=== Detection : pid file modified : memcached.127.0.0.1:11215
Register memcached repl node. Start monitoring. : (14090, 127.0.0.1:11215)
=== Checking memcached process (1/2) - pid : 14090 - addr : 127.0.0.1:11215 ... OK
=== Checking memcached process (2/2) - pid : 13972 - addr : 127.0.0.1:11215 ... NOK
Memcached process does not exist. It may abnormally terminate. : (13972, 127.0.0.1:11215)
Zookeeper error. zpath=/arcus_repl/group_list/test/g0/127.0.0.1:11215^127.0.0.1:20125^0000179165, error=-101(no node), arcus_memc_mon_rm_znode.c:313
Znode deletion fails.
Unregister memcached repl node. Stop monitoring. : (13972, 127.0.0.1:11215)
=== Detection : pid file deleted : memcached.127.0.0.1:11215
Memcached process exists. Do not unregister. : (-1, 127.0.0.1:11215)
Pid file deleted, but cannot unregister 127.0.0.1:11215 memcached node!
=== Checking memcached process (1/1) - pid : 14090 - addr : 127.0.0.1:11215 ... OK
=== Checking memcached process (1/1) - pid : 14090 - addr : 127.0.0.1:11215 ... OK
=== Checking memcached process (1/1) - pid : 14090 - addr : 127.0.0.1:11215 ... OK
=== Checking memcached process (1/1) - pid : 14090 - addr : 127.0.0.1:11215 ... OK

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.