Code Monkey home page Code Monkey logo

pravega-client-python's Introduction

Pravega Build Status codecov License Version CII Best Practices

Pravega is an open source distributed storage service implementing Streams. It offers Stream as the main primitive for the foundation of reliable storage systems: a high-performance, durable, elastic, and unlimited append-only byte stream with strict ordering and consistency.

To learn more about Pravega, visit https://pravega.io

Prerequisites

  • Java 11+

In spite of the requirements of using JDK 11+ to build this project, client artifacts (and its dependencies) must be compatible with a Java 8 runtime. All other components are built and ran using JDK11+.

The clientJavaVersion project property determines the version used to build the client (defaults to 8).

Building Pravega

Checkout the source code:

git clone https://github.com/pravega/pravega.git
cd pravega

Build the pravega distribution:

./gradlew distribution

Install pravega jar files into the local maven repository. This is handy for running the pravega-samples locally against a custom version of pravega.

./gradlew install

Running unit tests:

./gradlew test

Setting up your IDE

Pravega uses Project Lombok so you should ensure you have your IDE setup with the required plugins. Using IntelliJ is recommended.

To import the source into IntelliJ:

  1. Import the project directory into IntelliJ IDE. It will automatically detect the gradle project and import things correctly.
  2. Enable Annotation Processing by going to Build, Execution, Deployment -> Compiler > Annotation Processors and checking 'Enable annotation processing'.
  3. Install the Lombok Plugin. This can be found in Preferences -> Plugins. Restart your IDE.
  4. Pravega should now compile properly.

For eclipse, you can generate eclipse project files by running ./gradlew eclipse.

Note: Some unit tests will create (and delete) a significant amount of files. For improved performance on Windows machines, be sure to add the appropriate 'Microsoft Defender' exclusion.

Releases

The latest pravega releases can be found on the Github Release project page.

Snapshot artifacts

All snapshot artifacts from master and release branches are available in GitHub Packages Registry

Add the following to your repositories list and import dependencies as usual.

maven {
    url "https://maven.pkg.github.com/pravega/pravega"
    credentials {
        username = "pravega-public"
        password = "\u0067\u0068\u0070\u005F\u0048\u0034\u0046\u0079\u0047\u005A\u0031\u006B\u0056\u0030\u0051\u0070\u006B\u0079\u0058\u006D\u0035\u0063\u0034\u0055\u0033\u006E\u0032\u0065\u0078\u0039\u0032\u0046\u006E\u0071\u0033\u0053\u0046\u0076\u005A\u0049"
    }
}

Note GitHub Packages requires authentication to download packages thus credentials above are required. Use the provided password as is, please do not decode it.

If you need a dedicated token to use in your repository (and GitHub Actions) please reach out to us.

As alternative option you can use JitPack (https://jitpack.io/#pravega/pravega) to get pre-release artifacts.

Quick Start

Read Getting Started page for more information, and also visit sample-apps repo for more applications.

Running Pravega

Pravega can be installed locally or in a distributed environment. The installation and deployment of pravega is covered in the Running Pravega guide.

Support

Don’t hesitate to ask! Contact the developers and community on slack (signup) if you need any help. Open an issue if you found a bug on Github Issues.

Documentation

The Pravega documentation is hosted on the website: https://pravega.io/docs/latest or in the documentation directory of the source code.

Contributing

Become one of the contributors! We thrive to build a welcoming and open community for anyone who wants to use the system or contribute to it. Here we describe how to contribute to Pravega! You can see the roadmap document here.

About

Pravega is 100% open source and community-driven. All components are available under Apache 2 License on GitHub.

pravega-client-python's People

Contributors

anju-c-das avatar ankuranand11 avatar bhupender-y avatar gfinol avatar shwethasnayak avatar tkaitchuck avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pravega-client-python's Issues

Expose StreamCut API to start reading from certain position in stream

Problem description
If Client application crashes and if user wants to continue read the stream from where it left then at present there is no way to get StreamCuts from ReaderGroup.
StreamCuts are set of segment/offset pairs for a single stream that represent a consistent position in the stream.

Suggestions for an improvement
Expose get_streamcut() API from python client

Support async for write event

Problem description
Python write_event should have an async version. Currently there is a async version of read:

pub fn get_segment_slice_async<'p>(&self, py: Python<'p>) -> PyResult<&'p PyAny> {

We should add a similar "_async" version of write:
pub fn write_event(&mut self, event: &str, routing_key: Option<String>) -> PyResult<()> {

This will allow users to write multiple events without blocking.

Problem location
Add async version of write event in stream_writer.rs

Update release script to adopt Trusted publisher managment in pypi.

Problem description
Currently python release process required credentials to upload in pypi. Update release script to adopt Trusted publisher managment in pypi.

Problem location
Update the release script in tagPublish.yml so that it will not require user credentials for release.

Suggestions for an improvement
Update the release script in tagPublish.yml so that it will not require user credentials for release.

Update version to 0.3.7

Problem description
Update version to 0.3.7.

Problem location
Update the version from 0.3.5 to 0.3.7, so that we can prepare for new release.

Suggestions for an improvement

Code coverage improvements and reports for Python client

Problem description
No code coverage report been generated and published for python client.

Suggestions for an improvement
Configure the code coverage to generate and publish report for python client. Also add more code for improve the coverage upto 80%

GitHub action build fail for python client

Problem description
In github action we see below error

maturin failed
  Caused by: Error ensuring manylinux_2_35 compliance
  Caused by: Your library links libpython (libpython3.7m.so.1.0), which libraries must not do. Have you forgotten to activate the extension-module feature?
Traceback (most recent call last):
  File ".github/workflows/build_wheel.py", line 24, in <module>
    subprocess.run(command, check=True)
  File "/opt/hostedtoolcache/Python/3.7.17/x64/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['maturin', 'build', '--release', '--compatibility=manylinux_2_35', '--interpreter

Problem location
https://github.com/pravega/pravega-client-python/actions/runs/8044300967/job/21969199837

Suggestions for an improvement

Create an initial directory structure.

Problem description
Create an initial directory structure.

Problem location
Currently we have python client inside the rust client. Now we need to create a dedicated repo for python client.

Suggestions for an improvement
Create a dedicated repo for python client.

Update version to 0.3.8

Problem description
Update version to 0.3.8

Problem location
Update version from 0.3.7 to 0.3.8
This release includes newly created stream cut API

Cannot install pravega

Problem description

I'm trying to install pravega via poetry:

poetry install pravega

Using version ^0.3.8 for pravega

Updating dependencies
Resolving dependencies... (0.1s)

Package operations: 1 install, 0 updates, 0 removals

  • Installing pravega (0.3.8): Failed

BuildSystemTableValidationError

Failed to validate build-system in pyproject.toml: Unknown properties: classifiers, url, requires-python

at /opt/homebrew/Cellar/poetry/1.8.3_2/libexec/lib/python3.12/site-packages/build/_builder.py:113 in _parse_build_system_table
109│
110│ unknown_props = build_system_table.keys() - {'requires', 'build-backend', 'backend-path'}
111│ if unknown_props:
112│ msg = f'Unknown properties: {", ".join(unknown_props)}'
→ 113│ raise BuildSystemTableValidationError(msg)
114│
115│ return build_system_table
116│
117│

Cannot install pravega.

Obviously I'm using Python 3.12 and poetry. Latest macOS.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.