Code Monkey home page Code Monkey logo

ansj_seg's Introduction

Ansj中文分词(词库可热更新版)

研究目的

当前线上系统更新词库重启集群服务有风险,亟需一套词库热更新策略,保证在不停服务的前提下,增删用户自定义词库。

研究准备

研究思路

dataflow overview

实现步骤

  1. 在远程服务器Tomcat上部署词典文件
  2. 修改配合文件,写明远程词典地址
  3. 在程序启动是,创建定时线程任务ScheduledExecutorService,对应线程类Monitor
  4. 编写Monitor线程类,功能为发送HTTP请求并分析Last-Modify、ETags值是否变化
  5. 在DicLibrary类里,编写Monitor类监听到词库更新后,本地不停服务下载词典并更新词库的过程

实验截图

  1. 远程词典初始内容

screenshot01

  1. 启动主程序分析
  • 分析“明月几时有,把酒问青天”

screenshot02

  1. 词典增删改效果
  • 将“问青天”从远端词库删掉

screenshot03

  • 将“问青天”从远端词库增加

screenshot04

  • 将“问青天”修改为“问青”,由于不符合语言学模型,所以修改无效果

screenshot05

涉及到的源码

  1. [Modify] ansj_library.properties
  2. [Add] src/test/java/ TestRemoteDict.java
  3. [Modify] pom.xml
  4. [Add] src/main/java/org/ansj/monitor/Monitor.java
  5. [Modify] src/main/java/org/ansj/library/DicLibrary.java

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.