Code Monkey home page Code Monkey logo

spark-book-search's Introduction

spark-book-search

1. Приложение работает по принципу REST сервис с запросами вида localhost:8080/{book} , где book - это 
передаваемый аргумент названия текста
из папки scr/main/resources/books
2. Для обработки запросов используется класс PopularWordsServiceImpl.
                .flatMap((String line) -> WordsUtil.getWords(line).iterator())
                .filter(word-> !userConfig.excludeWords.contains(word))
                .mapToPair(word->new Tuple2<>(word,1))
                .reduceByKey((a,b) -> a + b);
        return output.collectAsMap();
Здесь привожу строки к нижнему регистру, разбиваю строки на отдельные слова при помощи WordsUtil, исключаю лишние
слова с помощью .filter (слова располагаются в user.properties) и далее кладу слова в Tuple и считаю их количество.
3.На контроллер данные возвращаются в виде Map. Сортирую значения и отправляю клиенту в виде json.

Пример запроса:
{"что":7797,"он":7491,"на":6780,"с":5932,"как":4112,"его":3955,"к":3469,"я":3103,"но":2777,"она":2752,"это":2567,
"было":2519,"то":2309,"а":2113,"сказал":2016,"так":1990,"о":1889,"же":1820,"за":1818,"все":1801,"по":1792,
"от":1773,"ему":1764,"ее":1700,"из":1695,"только":1616,"был":1584,...}

spark-book-search's People

Contributors

artbmstu avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.