The purple gradient is overlaid on top of the inner pages causing links to become unclickable.

For any given account, we need an endpoint to query the sourceSecondsDuration for all streams created thus far in an active billing cycle and at the end of the billing cycle.

sourceSecondsDuration for the active billing cycle will be displayed on the usage tracking page in the livepeer.com admin and must be updated at least daily. sourceSecondsDuration will also be used at the end of the billing cycle for billing via our Stripe integration. To get an accurate count, we will need to query for streams with a parent id for a given account.

The billing cycle is not the first of the calendar month to the end of the calendar month. The billing cycle should be able to be set arbitrarily.

What's the problem? (required)
From the Postgres refactor ticket:

There should really be a db[table].update({}) for partial updates, this will reduce the risk of different API servers clobbering the entire document with replace()

What's the current behavior? (required)
If a lot of different agents try and modify the same record at the same time, they'll stomp on each other.

All we need here is an update({}) method that only replaces a few fields of the SQL row instead of all of them; the semantics should be otherwise identical

I made a few last minute changes to get API things routing to the worker or upstream server as appropriate. https://github.com/livepeer/livepeerjs/blob/master/packages/api/src/app-router.js#L134

These are pretty ugly. Let's figure out a better way to configure this, probably by having a single list of path-to-regexp strings that control which route goes where.

Problem:
livepeer.com dashboard users are confused when the geo-located ingest and playback base URL changes overtime.

For reference, on a given stream page in the livepeer.com dashboard, this is how we display stream key, RTMP ingest URL and Playback URL:

Stream key: 1234-abcd-5678-efgh (hidden stream key to start)
RTMP ingest URL: rtmp://region-rtmp.livepeer.com/live/1234-abcd-5678-efgh
Playback URL: https://region-cdn.livepeer.com/hls/12344hh26rv8zj1b/index.m3u8

Solution:
Reformat these parameters in the UI to show ALL ingest and playback URL pairs.

Stream key: 1234-abcd-5678-efgh (hidden stream key to start)
Ingest and playback URL pairs:
RTMP ingest URL: rtmp://region-0-rtmp.livepeer.com/live/stream-key
Playback URL: https://region-0-cdn.livepeer.com/hls/12344hh26rv8zj1b/index.m3u8
RTMP ingest URL: rtmp://region-1-rtmp.livepeer.com/live/stream-key
Playback URL: https://region-1-cdn.livepeer.com/hls/12344hh26rv8zj1b/index.m3u8
RTMP ingest URL: rtmp://region-2-rtmp.livepeer.com/live/stream-key
Playback URL: https://region-2-cdn.livepeer.com/hls/12344hh26rv8zj1b/index.m3u8

Notes:
The ingest and playback base URLs should be grouped as pairs to help emphasize to the user that they are related. The documentation will also specify that you cannot use a region-1 ingest URL with a region-2 playback URL.

Ties into convo about hubspot

What's the problem? (required)
We need to figure out how to route incoming playback requests to horizontally-scaled ingest servers. Two use cases in mind here:

1. Routing incoming playback requests to Mist servers that automatically route to an origin
2. Routing "custom names" to go-livepeer/Mist, allowing for "vanity names" like /stream/eli.m3u8

Here's what I'm thinking so far.

• The API server gains the capability to automatically create Traefik IngressRoute resources that route e.g. /stream/eli* to the prod-livepeer-{shard} service or /hls/{playbackId} to the mist-server-0 service.
• The first place this will happen is on mist-api-connector's /stream/:id/setactive endpoint. When a new Mist stream comes in, we will create a new route for /hls/{playbackId} to the appropriate Mist server. mist-api-connector will be provided with the pertinent service name that it can then pass to the API.
• Later on this can also happen in the /api/stream/hook endpoint so streams with "vanity names" can be implemented with go-livepeer.

Alternatives:
This creates a dependency on the availability of the local Kubernetes API for ingesting new streams. We could instead keep the name --> service mapping in our own API database and proxy the streams through the API. This would require us to write some manner of API-aware proxy and send all traffic through it. I like using Traefik instead, but it's important to keep the tradeoff in mind.

Describe the bug (required)
A new stream in livepeer.com is created every time a new Mist stream is created. However, the status of this livepeer.com stream stays idle.

Expected behavior (required)
The status of the livepeer.com stream should say active.

To Reproduce (required)
Steps to reproduce the behavior:

1. Create a stream in Mist with Livepeer transcoding and stream into it. This should result in a new Livepeer stream.
2. In livepeer.com, verify the stream is created.
3. The stream status says "idle" even though the stream is active.

Note - I don't think this bug is super critical right now because not a lot of people are using livepeer.com AND mist at the same time, but we should keep track of it here and fix it at some point.

Describe the bug (required)
Forgot password email gives a 404

Expected behavior (required)
Forgot password workflow allows the user to reset the password.

To Reproduce (required)
Steps to reproduce the behavior:

1. Go to livepeer.com
2. Enter email in the "forgot password" form
3. Click on the link sent in the email
4. You get a 404

Problem
Above is roughly what the current implementation looks like. This list of ingest and playback URLs is confusing for users still. It isn't clear how to pick among MDW, FRA and future ingest points, nor is it clear that you must use each ingest/playback URL as a pair.

Solution

1. After the Ingest and Playback URL Pairs: section header, add an italicized link to a new documentation page. The link text is "Learn how to pick an ingest and playback URL pair." and it will link to a new documentation page at /docs/guides/dashboard/ingest-playback-url-pair. The link should open in a new tab.

2. Reformat the list of ingest and playback URL pairs.

   • Add numbers after each pair
   • In the state before the "Show secret stream key" button is clicked, remove "Reveal your stream key and the full URL via the button above." next to all the RTMP ingest URLs.
   • Add padding below each pair.
   • Add padding above the Ingest and Playback URL Pairs: section header.

Below is roughly what the updated section should look like after the "Show secret stream key" button is clicked.

Necessary for us to survive crashes of the Kubernetes control pane, which we should be able to do.

What's the problem? (required)
Sometimes the encoding software doesn't do a good job creating the RTMP ingest URL (for example, Wirecast). Especially when the user is copy/pasting the URL, it's easy to end up with something like rtmp://mdw-rtmp.livepeer.com/live//{stream-key}. In this case, our API doesn't accept the stream.

What's the current behavior? (required)
Our API should recognize this case and still accept the stream.

We don't currently have a mechanism for listing things by date - this will be nice for stream status so you see the most recent ones instead of the (potentially) thousands of old streams that don't matter anymore.

Executing this correctly will mean that we can likely generate API clients in other languages too, and that rules.

Currently, requests for https://livepeer.live/stream/UUID.m3u8 hit every single broadcaster to amalgamate manifests. This is inefficient and fragile; we might break if a single broadcaster request gets stuck.

What we should do instead is maintain a mapping of stream UUID to broadcasters and only amalgamate from broadcasters that have data for us. To accomplish this, we can create a new record for a broadcaster when we get an incoming webhook request, and then only make upstream requests to the pertinent broadcaster.

Some kind of unhandled promise rejection. Probably need better error handling there.

Update the following docs pages:

1. /docs/guides/api/base-urls
2. /docs/guides/dashboard/stream-page-specifications
3. /docs/guides/dashboard/broadcast-a-stream-session
4. /docs/guides/dashboard/playback-a-stream

What's the problem? (required)
We want to be able to limit bandwidth used by API users.

Proposed implementation
Mist allows to configure webhook that will be triggered when stream exceed some threshold -
we can use that to implement configurable system for limiting bandwidth.

Bandwidth could be constrained per stream and per user (say, we have constrain of 10Mb per stream and 50Mb per user - that means that user will not be able to simultaneously transcode more than 5 streams).
There will be global limit that can be overridden per user.

API changes:
add global object
/globalConfig

{
    bandwithLimit: {
        stream: number, // bytes per second
        user: number // bytes per second
    }
}

Also field bandwithLimit with the same structure could optionally be added to user object.
API endpoint /api/stream/key/:streamKey which is used by mist-api-connector to retrieve
stream's information will also return bandwithLimit inside stream object. First it will
look for bandwithLimit in the user's object, and if not set then will return bandwithLimit
from the globalConfig object. mist-api-connector will use that information to stop streams
that exceed specified threshold.

This happens a lot and I don't know why

@iameli commented on Mon Apr 06 2020

These endpoints all return cluster-specific or geo-located information about broadcasters and orchestrators.

Basically this looks like:

1. Compile everything into a single file (yarn run prepare:worker)
2. Boot up that worker with @dollarshaveclub/cloudworker
3. Run the full test suite with that as the server, the same way we do with Level and Postgres and such so far
4. Shut down the server

Part 1: Wildcard streams

Currently, UGC platforms building on top of the Livepeer API would have to create a new Stream object for each user. That's not ideal; if I want to update my bitrate ladder site-wide, I'd have to iterate through every single stream definition! So, let's allow one stream to be used multiple times.

Streams get a new field: {"wildcard": true/false}.

If false, current behavior:
Ingest URL: rmtp://${ server }/live/${ streamKey }
Playback URL: https://${ cdn }/hls/${ playbackKey }/index.m3u8 All is well.

If true, each streaming endpoint can be used multiple times. (And probably streaming to the root endpoint is not allowed.)
Ingest URL: rtmp://${ server }/live/${ streamKey }+${ anything }
Playback URL: https://${ cdn }/hls/${ playbackKey }%2B${ anything }/index.m3u8

To implement this part, all we need to do is add the field on the server then tweak the logic on mist-api-connector to handle URLs in such a form.

Part 2: Playback configuration webhook

That alone will be sufficient for lots of use cases, but there's also a problem for the primary UGC use case... there's only one secret streamKey. if they use ingest URLs of the form rtmp://example.com/live/streamKey+userName, that means anyone can stream to any user by using rtmp://example.com/live/streamKey+otherUserName!

The solution here is to allow for UGC platforms to provide internal stream keys to their users and then provide the playbackId to Livepeer during the resolution of the streamStarted webhook. In this example, imagine I'm implementing a service called Stream.Town. The workflow will look like this:

1. Before anyone streams, I create a Stream ` object with my desired renditions. It has:

POST /api/stream
{
   ...rendition configuration omitted
  "wildcard": true
}

201 Created
{
  "id": "livepeer-id",
  "streamKey": "livepeer-stream-key",
  "playbackId": "livepeer-playback-id"
}

2. I also create a webhook for the streamStarted event of that stream:

POST /api/webhook
{
  eventType: "streamStarted",
  streamId: "livepeer-id",
  url: "https://stream.town/api/handle-webhook"
}

2. A user signs up on Stream.Town. When they sign up, I provide them with a stream key and save it in their user record in the Stream.Town database:

{
  "username": "iameli",
  "streamKey": "streamtown-stream-key"
}

3. On the StreamTown frontend, I geolocate (GET /api/ingest) then provide them with their RTMP ingest URL: rtmp://some-origin.livepeer.com/live/livepeer-stream-key+streamtown-stream-key.
4. The user streams in. A series of webhooks fire, and stream.town returns the playbackId during the streamStarted event.

POST https://stream.town/api/handle-webhook
{
  "stream": {
    "id": "livepeer-id",
    "playbackId": "livepeer-playback-id"
    ...
  }
  "streamKey": "streamtown-stream-key"
}

I proceed to check my database for streamtown-stream-key, and find that it corresponds to the user iameli. So I send that back from the webhook:

200 OK
{
  "playbackId": "iameli"
}

5. That playbackId gets inserted into the newly-created Stream Session as livepeer-playback-id+iameli and used for playback at https://example-cdn.livepeer.com/hls/livepeer-playback-id%2Biameli/index.m3u8.

Problem:
The table looks broken. Also, we no longer need a recording URL column. See below.

Solution
Remove the table for now.

This is the HTTP video ingest endpoint. Technically this isn't on the API, it's on broadcasters, but I think we'd like it to be on the API at some point. Regardless, this is a good place to document the endpoint and its quirks such as the content-duration and content-resolution headers.

Here's an example of using it:

curl -XPOST -d @8.ts -H "authorization: Bearer 2de47e83-9c9f-4d2e-a9f1-7c798a01ddc3" -H "content-duration: 2030" -H "content-resolution: 1280x720" https://aks-default-16253462-vmss000000.livepeer.live/live/c25cb445-34df-44b6-aa4b-a5e0a4c207d3/8.ts

Problem:
API users are confused when the geo-located ingest and playback base URL changes overtime.

Solution:
Return all ingest and playback base URLs. By calling https://livepeer.com/api/ingest, you should get results like this:

[
  {
    ingest: "rtmp://{region-0}-rtmp.livepeer.com/live",
    playback: "https://{region-0}-cdn.livepeer.com/hls"
  },
 {
    ingest: "rtmp://{region-1}-rtmp.livepeer.com/live",
    playback: "https://{region-1}-cdn.livepeer.com/hls"
  },
 {
    ingest: "rtmp://{region-2}-rtmp.livepeer.com/live",
    playback: "https://{region-2}-cdn.livepeer.com/hls"
  }
];

Notes:
The ingest and playback base URLs should be grouped as pairs to help emphasize to the user that they are related. The documentation will also specify that you cannot use a region-1 ingest URL with a region-2 playback URL.

Describe the bug (required)
The output video segment length from livepeer.com is not close to 2 seconds

Expected behavior (required)
The Livepeer transcoders tries to create uniform segment lengths of 2s chunks

To Reproduce (required)

1. Create a stream in livepeer.com
2. Stream into the ingest using ffmpeg and the playdj.mp4 input file: ffmpeg -re -i https://eric-test-livepeer.s3.amazonaws.com/playdj.mp4 -c copy -f flv rtmp://mdw-rtmp.livepeer.com/live/{stream-key}
3. Curl the output video playlist
4. The output segments range from 8 to 10 seconds

Describe the bug (required)
On mobile the gradient on the home page hero flickers.

Screenshots (optional)

We need one!

Describe the bug (required)
The /geolocate API call gets stuck when a region goes down. This happened when the EWR data center went down on 10/27.

Expected behavior (required)
The /geolocate API call should timeout for the location that doesn't return in time.

Note
This fix should be deployed to the Cloudflare worker AND the data center specific geolocate endpoint.

1. New flag, adjacent to --postgres-url: --postgres-replica-url. It can be pointed at a read replica of the primary database.
2. If set, initialize a second Pool object, this.replicaPool, alongside this.pool in db.ts.
3. In db.ts, add a new function adjacent to query() called replicaQuery() that can be called in exactly the same way. If this.readPool is unset, it does exactly the same thing as query(). If it is set, it performs the request against this.replicaPool instead of this.pool.
4. Update the find() and get() functions in table.ts with a new option useReplica that defaults to true. (get() will need to add a second parameter for options entirely.) This option determines whether they call this.db.query() or this.db.replicaQuery(). The idea here is that we want to hit the read replica almost all of the time but if you're in a situation where you absolutely must have the most up-to-date data you can set {useReplica: false} and sacrifice latency for correctness.

• Response codes 2xx vs 4xx vs 5xx
• Database latency

1. On /app/user/plans and /pricing, change the row order top to bottom to: Transcoding, Recording Storage, Streaming
2. In the pricing chart, on the Recording Storage row for Personal add "None"
3. In the pricing chart, on the Streaming row for Personal, change "Up to 10 concurrent viewers" to "Up to 10 concurrent viewers / account". The per account is important.
4. In the pricing chart, on the Transcoding and Streaming rows for Pro, change "$0.005" to " $0.005 USD" and "$0.01" to "$0.01 USD" respectively.
5. When I create a new account, verify, navigate back to the tab telling me to check my email /app/user/verify, click "home" (the livepeer icon in the upper left corner), and then click the dashboard button, I don't go to /app/user/ but incorrectly to /app/user/verify.
6. The sessions table is all messed up. Remove it. The bug is in prod too. Already ticketed

And of course, we need migration plan for existing users to move to the Personal Tier.

The newsletter should get posted to the following: https://livepeer.us16.list-manage.com/subscribe/post?u=57807e9b74db375864b2c4c68&id=5b12d9c158.

We can use the same solution as the .org. https://github.com/livepeer/livepeer-org/blob/master/components/sections/footer/prefooter-banner.tsx#L40

Here are the three different videos we'd like to feature on the home page hero. The idea is to serve a different one at random on page load.

hero_videos.zip

Describe the bug (required)
When create a new stream with the segment transcoding API, some of the values passed back cannot be used. These include:

• streamKey
• playbackId
• userId
• renditions
• kind

Expected behavior (required)
The API response should only contain the necessary information for a segment transcoding API user.

To Reproduce (required)
Steps to reproduce the behavior:

1. POST to the livepeer.com/api/stream endpoint