Code Monkey home page Code Monkey logo

libltp's Introduction

libltpbeta

本仓库用于为 Language Technology Platform 创建不同语言的绑定。

  • Rust
  • C++
  • Java
  • Python

模型地址

模型 大小 SHA256
Base 491.9MB 2df2575325e740e894ba865c74ef81498a5ffaabca58626e977dc5dc95cb2808
Small 156.8MB 85db4bcf6b1df8f0a0d44466050587229f294a4134c525c167942786052c292d
Tiny 31.3MB e662eae1ed4c904eeefe33534526bf05c007a44187be52e0f47b5c2f78b7e29d

C++

cd ltp-cpp
mkdir build && cd build
cmake -DENABLE_LTO=ON -DCMAKE_BUILD_TYPE=Release ..
make

Java

cd ltp-java/rust && make lib
cd .. && gradle build

Python

cd ltp-py
maturin build -i python3

TODO

  • 可选任务
  • 语言词语对齐
  • 其他语言支持 cuda
  • 自动下载模型

开源协议

  1. 语言技术平台面向国内外大学、中科院各研究所以及个人研究者免费开放源代码,但如上述机构和个人将该平台用于商业目的(如企业合作项目等)则需要付费。
  2. 除上述机构以外的企事业单位,如申请使用该平台,需付费。
  3. 凡涉及付费问题,请发邮件到 [email protected] 洽商。
  4. 如果您在 LTP 基础上发表论文或取得科研成果,请您在发表论文和申报成果时声明“使用了哈工大社会计算与信息检索研究中心研制的语言技术平台(LTP)”. 同时,发信给[email protected],说明发表论文或申报成果的题目、出处等。

libltp's People

Contributors

alongwy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

libltp's Issues

ltp-java编译出错

我在使用ltp-java时编译报错

本地环境:Windows 11 专业版 版本:22621.1265

  • java 信息:
[D:\java\ltp\libltp\ltp-java]$ java -version
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
  • cargo 信息:
[D:\java\ltp\libltp\ltp-java]$ cargo --version
cargo 1.68.1 (115f34552 2023-02-26)
  • gradle 信息:
[D:\java\ltp\libltp\ltp-java]$ gradle -v

------------------------------------------------------------
Gradle 8.0.2
------------------------------------------------------------

Build time:   2023-03-03 16:41:37 UTC
Revision:     7d6581558e226a580d91d399f7dfb9e3095c2b1d

Kotlin:       1.8.10
Groovy:       3.0.13
Ant:          Apache Ant(TM) version 1.10.11 compiled on July 10 2021
JVM:          1.8.0_121 (Oracle Corporation 25.121-b13)
OS:           Windows 10 10.0 amd64

我从仓库中把整个项目clone下来,

  • 本地目录结构:
[D:\java\ltp\libltp\ltp-java]$ tree ./
卷 DATA 的文件夹 PATH 列表
卷序列号为 00000041 AE2A:AF4A
D:\JAVA\LTP\LIBLTP\LTP-JAVA
├─.gradle
│  ├─8.0.2
│  │  ├─checksums
│  │  ├─dependencies-accessors
│  │  ├─executionHistory
│  │  ├─fileChanges
│  │  ├─fileHashes
│  │  └─vcsMetadata
│  ├─buildOutputCleanup
│  └─vcs-1
├─build
│  ├─classes
│  │  └─java
│  │      ├─main
│  │      │  └─cn
│  │      │      └─edu
│  │      │          └─hit
│  │      │              └─ir
│  │      └─test
│  │          └─cn
│  │              └─edu
│  │                  └─hit
│  │                      └─ir
│  ├─generated
│  │  └─sources
│  │      ├─annotationProcessor
│  │      │  └─java
│  │      │      ├─main
│  │      │      └─test
│  │      └─headers
│  │          └─java
│  │              ├─main
│  │              └─test
│  ├─libs
│  ├─reports
│  │  └─tests
│  │      └─test
│  │          ├─classes
│  │          ├─css
│  │          ├─js
│  │          └─packages
│  ├─test-results
│  │  └─test
│  │      └─binary
│  └─tmp
│      ├─compileJava
│      ├─compileTestJava
│      ├─jar
│      └─test
├─gradle
│  └─wrapper
├─rust
│  └─src
└─src
    ├─main
    │  ├─java
    │  │  └─cn
    │  │      └─edu
    │  │          └─hit
    │  │              └─ir
    │  └─resources
    └─test
        ├─java
        │  └─cn
        │      └─edu
        │          └─hit
        │              └─ir
        └─resources

  • 执行脚本
D:\java\ltp\libltp>cd ltp-java/rust && make lib
cargo build --release
    Updating git repository `https://github.com/AlongWY/onnxruntime-rs.git`
    Updating crates.io index
   Compiling winapi v0.3.9
   Compiling proc-macro2 v1.0.54
   Compiling quote v1.0.26
   Compiling memchr v2.5.0
   Compiling libc v0.2.140
   Compiling windows_x86_64_msvc v0.42.2
   Compiling log v0.4.17
   Compiling syn v1.0.109
error: linker `link.exe` not found
  |
  = note: program not found

note: the msvc targets depend on the msvc linker but `link.exe` was not found

note: please ensure that Visual Studio 2017 or later, or Build Tools for Visual Studio were installed with the Visual C++ option.

note: VS Code is a different product, and is not sufficient.

error: could not compile `windows_x86_64_msvc` due to previous error
warning: build failed, waiting for other jobs to finish...
error: could not compile `log` due to previous error
error: could not compile `quote` due to previous error
error: could not compile `memchr` due to previous error
error: could not compile `syn` due to previous error
error: could not compile `proc-macro2` due to previous error
error: could not compile `libc` due to previous error
error: could not compile `winapi` due to previous error
make: *** [lib] Error 101
D:\java\ltp\libltp\ltp-java\rust>cd .. && gradle build

> Task :test FAILED

LTPTest > test() FAILED
    java.lang.ExceptionInInitializerError at LTPTest.java:12
        Caused by: java.lang.NullPointerException at LTPTest.java:12

1 test completed, 1 failed

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///D:/java/ltp/libltp/ltp-java/build/reports/tests/test/index.html

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1s
4 actionable tasks: 1 executed, 3 up-to-date

我还在入门,不太懂这个,请帮忙看看。
谢谢 (^_^)

请问目前libltp支持单独分词吗

   你好,我在使用过程中想只使用分词功能,看目前的pipeline_batch() 方法调用了所有任务。通过查看底层代码,

发现onnxruntime-rs.git?branch=al/dyn+cuda 这个分支提供session 的 run_with_options 方法,
通过以下方法发生段错误,这是onnx 不支持计算部分子图吗? 还是其他原因,谢谢。
let name_seg = String::from("seg");
let output_type_seg = TensorElementDataType::Int64;
let dimensions_seg: Vec<Option> = vec![None, None];
let output = Output{ name: name_seg,
output_type: output_type_seg,
dimensions : dimensions_seg
};
let output_names: Vec = vec![output];
println!("before onnx run with option");
println!("{:?}", output_names);
let mut result = self.session.run_with_options(input_arrays, &output_names)?;

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.