Code Monkey home page Code Monkey logo

hfc-pubsub's Introduction

Google Pub/Sub Node.js Framework

npm Node.js CI Codacy Badge Codacy Badge dependencies Status Known Vulnerabilities

PubSub is a lightweight framework and subscription server for Google Pub/Sub. It was created to speed up development time โšก๏ธ and it provides a common foundation for building event driven applications. It lets developers define topics and subscriptions simply and declaratively, while additionally offering a simple subscription server to run all of a project's subscription handlers.

demo of subscription service starting through cli

Features

PubSub Framework is built with the developer experience in mind.

  • Built with ๐Ÿ’š and Typescript

    • In built types
    • Run with ts-node in dev
  • Easy to setup

    • Define pub/sub subscriptions and topics in a declarative way
    • Define your subscription handlers with a simple object
    • Define a topic and publish messages with a few lines of code
  • โœ‚๏ธ Developer experience

    • Start receiving messages in minutes
    • Add configuration defaults for all your subscriptions in one place
    • DLQs and IAM automatically handled for you
    • Auto load code from folder

Links

Getting started

Moved to Docs

CLI commands - starting and listing subscriptions

Moved to CLI

Subscriptions

Moved to Subscriptions

Topics

Moved to Topics

Subscription Service

Moved to Service

Enabling Synchronous Driver

Moved to Drivers

hfc-pubsub's People

Contributors

anthonymartin avatar behnamghafary avatar caglarsayin avatar dependabot[bot] avatar github-actions[bot] avatar hfc-bot avatar ishan123456789 avatar ketan-saxena avatar mzmmlds avatar rndd avatar rohit-gohri avatar samrocksc avatar vinodrnair avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

hfc-pubsub's Issues

Allow bypassing of topic creation on subscription start

Summary

Sometimes you just want to create a subscription to another topic, but that topic already exists or you do not want the topic created with the subscription. If you want to define your topics in something like terraform, where you can control access to it in code.

Current Behavior

Currently if the DLQ is enabled, it will auto create the topic.

Desired Behavior

If we could add a flag that would allow the subscriber to simply fail and throw if the topic does not exist yet....or have a failure method where we can pass through the error, or handle it gracefully.

Message `.toJSON()` method is unsafe

Summary

I suppose it's not a big deal, but for me using message.toJSON() method was a bit confisung since it was unsafe. I think, if it exists and is represented in documentation, maybe we don't want it to crash the whole app=)

Current Behavior

App crashes completely if I publish invalid json to PubSub topic and do message.toJSON() in my code. This doesn't go to handleError, just causes crash and process termination.

Desired Behavior

I think this method should use try-catch internally and return undefined on parsing error. I can submit a pr with this change.

Allow feature to Quick Post on a topic

Summary

Being able to only make one round trip to Google Pub/Sub's API surface is important on some services. Everytime we initiate a topic, we have to check if it exists. Being able to allow the user of the library to memoize whether a topic exists or not can be important.

The current behavior is safe, and allows for a failsafe to make sure a topic exists before posting.

Current Behavior

Currently when initializing the Topic class, it does an .exists() check, which causes a round trip to the Google Pubsub api surface.

Requested Behavior

We have a way to expedite this by simply being able to call .publish() only once. Maybe a flag, or having a Public method on the class we can call instead of having to call new on the hfc pubsub package.

Informative Error Handling

Overview

As a developer, I would like to receive informative error handling about which subscription or topic I am failing to subscribe to.

Current Behavior

Currently when there is an issue on the startup of a subscription server we will simply receive something like this:

Error: 7 PERMISSION_DENIED: User not authorized to perform this action.
    at Object.callErrorFromStatus (/usr/src/app/node_modules/@grpc/grpc-js/build/src/call.js:31:26)
    at Object.onReceiveStatus (/usr/src/app/node_modules/@grpc/grpc-js/build/src/client.js:179:52)
    at Object.onReceiveStatus (/usr/src/app/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:336:141)
    at Object.onReceiveStatus (/usr/src/app/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:299:181)
    at /usr/src/app/node_modules/@grpc/grpc-js/build/src/call-stream.js:145:78
    at processTicksAndRejections (internal/process/task_queues.js:79:11)

Desired Behavior:

Along with the actual error message, perhaps an appended key for which Subscriber/Topic that this happened for. In node you have the ability to actually extend the error class, so perhaps we could extend it to pass in a custom error message?

Remove dependency on subscription folder

Summary

Sometimes, it makes sense to use the same codebase, but to load it with multiple environments to start different subscription.service.ts files. This behavior is currently supported with the work around of creating a subfolder in your desired subscriptions folder /subscriptions/index.ts(or whatever you would like to name it).

** Note that you must include a filename above in order to trick the typescript compiler into creating the subscription/ folder, you cannot use an empty folder.

This is a bit of a work around that we can improve I do believe!

Current Behavior

Currently when providing PUBSUB_ROOT_DIR=./dist/pubsub/xxx, the subscription loader requires that a subfolder /subscriptions exists. Without this folder, an error is thrown upon npx subscription list/start.

Desired Behavior

We can simply point PUBSUB_ROOT_DIR to whatever folder we want, and it will not force you to have a subcriptions folder.

Use default topic.publish Attribute Type

Overview

I would like to take advantage of all features the google pubsub topic publish feature has to offer.

Current Behavior

Currently we have a stricter than default layer over thetopic.publish(message, config) that does not allow fore features such as publishing with topic attributes(e.g. https://github.com/googleapis/nodejs-pubsub/blob/master/samples/publishMessageWithCustomAttributes.js)

Desired Behavior

We can simply use the default interface for the exported type: Attributes found here: https://github.com/googleapis/nodejs-pubsub/blob/master/src/publisher/index.ts#L33-L35

While this is a simple key type, it is non-restrictive, and will automatically allow us to take advantage of new publish features in the future instead of having to add them manually to our current custom interface for topics.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.