Code Monkey home page Code Monkey logo

xiedudu's Projects

bdp icon bdp

A prototype project of big data platform, the source codes of the book Big Data Platform Architecture and Prototype

django icon django

The Web framework for perfectionists with deadlines.

hdfsfilemerge icon hdfsfilemerge

【HDFS小文件合并】功能介绍： HDFS文件块大小为128M,长期使用集群使的文件块过多，小数据文件过多造成集群报警。因此需在对小文件进行合并处理。 注意以下任意参数不可缺失，程序中不设默认参数。 入口函数： com.surq.hdfs.file.merge.Main 参数格式： --inFileType ..........欲合并文件的类型，支持文本类型和压缩格式的gzip(.gz)文件;设为all：压缩input的下的所有文件(spark直接读入路径)。 --outFileType ........合并结果文件的类型,gz:压缩格式;txt:文本格式(除gz之外所有参数全部认为是文本格式)。 --limitFileSize .......压缩结果文件的上限大小,单位M比如：128 --input ...............欲合并文件的路径。 --output ..............合并结果的输出路径。 --ismove...............是否把生成的结果移动到输入路径（input）中，设为true时转移完成后会将output路径删除。 --isdelete.............合并成功后是否把原始数据(input下的欲合并文件)删除。 --toEmail .............报警邮件接收人地址用逗号隔开,设为none将不发送邮件。

incubator-superset icon incubator-superset

Apache Superset (incubating) is a modern, enterprise-ready business intelligence web application

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.