In #150, @domenic wrote the following. Creating an issue so it doesn't get lost.

Note that (if it has multi-implementer interest) the https://wicg.github.io/picture-in-picture/#htmlvideoelement-extensions section should migrate to the HTML Standard, linking to the algorithms defined in this separate specification. This would also allow the chance to fix several issues with the current specification due to insufficient integration (e.g. the use of EventHandler attributes on HTMLMediaElement instead of GlobalEventHandlers, or the way that autoPictureInPicture, disablePictureInPicture, onenterpictureinpicture, andonleavepictureinpicture have no definitions).

In https://github.com/WICG/picture-in-picture/pull/112/files#r249006604, it was agreed to provide a clear definiton of autoPictureInPictureElement.

Background: #41

In the demo I am working on, we want to stop displaying video (and display some message) when the window is hidden. However, if we're pipped, we can't tell when we're minimized, because PIP always causes the visibility API to return 'visible'.

Would it be possible to add another property to the page visibility API to indicate PIP mode? This would allow a page using PIP to understand if its main document is still visible, which seems like something it would want to know.

See also #80.

Some browsers may have some kind of "auto-pip" behavior such as Chrome on Android O. It would be nice to have a way to prevent this behavior with the Picture In Picture Web API.

It could be an HTML attribute similar to disableRemotePlayback:

<video disableAutoPictureInPicture>

Or an event listener we could use to prevent element to enter pip:

video.addEventListener('beforepictureinpicture', event => {
  event.preventDefault();
});

What do you think @mounirlamouri @jernoble?

The explainer.md file says that

The API will allow the website to check if Picture in Picture can be triggered for a given video.

However, the API only exposes document.pictureInPictureEnabled which tells only if Picture in Picture can be triggered, but not for a given video.

As the API only applies on HTMLVideoElement at the moment but is meant to be extensible, I'd rather update the sentence to "The API will allow the website to check if Picture in Picture can be triggered".

What do you think @mounirlamouri?

@tidoust Can you reconfigure PR Preview at https://github.com/w3c/picture-in-picture/settings/installations?
We've lost it when migrating to W3C org.

Hello. I'd love to be able to make a canvas or HTML content PIP. I had a remote spectrum analyzer project that renders in SVG, for example, that would be great to keep on screen while doing other work on my device.

I saw the current draft has stated some constraints but not all of them, does this API need some more constraints such as:

• Max PiP overlay window width & height, as an how big a user is allowed to resize their PiP window?
• Triggered by action only so ads & spammy dialog's can not pop-up on page load.
• The recommendation fo the status of original window audio, video, keyframe animation, etc when they get teleported to the overlay window.
• etc..

Some might be implementation details, but still would be good to discuss at TPAC what needs to go in spec and what not.

Implementing this now and I'm assuming the event will not bubble nor be cancellable but having some writing about it would be nice :)

Because in some browsers, when the mouse leaves the PIP, the control will always exist, I need to hide the play button, NiMo Tv(https://www.nimo.tv/) is treated like this, but I don't know how he implemented it, can someone help me? Thank you

I believe we should let the UA (or OS) pick the preferred X and Y positions for the the PiP window. I'm not sure a website would know better than the OS what is better given a current setup (windows, workspaces).

I've added the stuff needed by the w3c. Please note the index.html (along with some other magic files) that was added to the root of your repo. You can delete it an use bikeshed if you wish.

"Add a fullscreen option in PIP window " is the option already available I've tried to trigger fullscreen when the user closes the pip window but obviously it is going to throw an error
Failed to execute 'requestFullscreen' on 'Element': API can only be initiated by a user gesture.

What does happen to PictureInPictureWindow object when exitPictureInPicture?
Should it be set to null?
Should its width and height dimensions reset? If so, would 0 makes sense?

What do you think @mounirlamouri?

Currently the PiP implementations are very restricted. Will there be support for basic controls like volume button/mute etc?

In Chrome and Safari, a video element not attached to the DOM can start playing and enter Picture-in-Picture. When video is attached to the DOM while playing in Picture-in-Picture, browser implementations differ though when video is removed from the DOM:

• In Chrome, it pauses the video as HTML spec says and nothing more.
• In Safari, it pauses the video as well but also leaves Picture-in-Picture.

It looks like Safari takes a hint from the Fullscreen API:

Whenever the removing steps run with a removedNode, run these steps:

1. Let document be removedNode’s node document.
2. Let nodes be removedNode’s shadow-including inclusive descendants that have their fullscreen flag set, in shadow-including tree order.
3. For each node in nodes:
   1. If node is document’s fullscreen element, exit fullscreen document.
   2. Otherwise, unfullscreen node.
   3. If document’s top layer contains node, remove node from document’s top layer.

Note that the Fullscreen API doesn't allow an element not atttached to the DOM to enter fullscreen unlike Picture-in-Picture that does not prevent it.

Just a minor thing, but in:

Queue a task to fire an event with the name enterpictureinpicture at the video with its bubbles attribute initialized to true and its pictureInPictureWindow attribute initialized to Picture-in-Picture window.

According to DOM's specs rules for firing an event, you should be. passing the Interface name to use there (i.e., EnterPictureInPictureEvent).

From DOM:

To fire an event named e at target, optionally using an eventConstructor, with a description of how IDL attributes are to be initialized, and a legacy target override flag, run these steps:

So, something more like:

Queue a task to fire an event with the name "enterpictureinpicture", using {{EnterPictureInPictureEvent}}. at the video with its bubbles attribute initialized to true and its pictureInPictureWindow attribute initialized to Picture-in-Picture window.

Just a small thing... when talking about content attributes VS IDL attributes, make sure you use this syntax for content attributes:

<{element/attribute}>

So:

<{video/disablePictureInPicture}>

Interactive arbitrary content will have limited keyboard input allowed. Developers should have a a way to detect this. In development this could be a console message.

On desktop, there's a couple of meaningful implementations/use cases for PIP. One is traditional TV style PIP where you get an always-on-top window that remains topmost at all times. However, a number of existing sites use pop-out windows which are non-topmost. There are some positive properties of this; it's possible to watch content without taking up a browser-sized window, but if you do switch to larger app windows they're not obscured by the video.

Since I believe we're encouraging sites to deprecate legacy approaches like pop-out windows, it might be nice to allow them to request less intrusive non-topmost behavior (e.g via options). Alternately, if both topmost and non-topmost are relevant, we might leave control with this purely with the UA (e.g. via some kind of "pin" affordance that toggles topmost status).

Currently, these are provided as options when PIP is invoked, but is it clear whether/how dimensions are exposed? There's a few things related to this:

• For users who regularly view using PIP, sticky position/size would be nice. Do we generally expect UAs to do this, or as with something like mute do we expect the site to persist this across API invocations? If the latter, how do sites read this information so that they can persist it?
• Do we expect to expose any control over whether users can resize PIP windows? (On desktop this seems particularly useful).
• If users can resize PIP windows, how do we expose this to the page? ABR logic that decides among multiple possible encodes with different resolutions often depends on this.
• How do we handle multiple displays? Especially if they have different properties (e.g. DPI) that affect content rendering?

Seems a bit weird to have "Enter" at the front of EnterPictureInPictureEvent... why not just generalize it to "PictureInPictureEvent"?

@mounirlamouri suggested offline we may want to allow video to enter Picture-in-Picture without user activation if there's already a visible PiP window in the context of the document, in other words if document.pictureInPictureElement is set.

  The {{requestPictureInPicture()}} method, when invoked, MUST
  return <a>a new promise</a> |promise| and run the following steps <a>in
  parallel</a>:

  1. Let |video| be the video element on which the method was invoked.
- 2. Let |userActivationRequired| be `true`.
+ 2. Let |userActivationRequired| be `true` if {{pictureInPictureElement}} is
+     null. It is `false` otherwise.
  3. Let |playingRequired| be `false`.
  4. Run the <a>request Picture-in-Picture algorithm</a> with |video|,
      |userActivationRequired|, and |playingRequired|.

This could help in situations where there are multiple video elements for one PiP media session.

What do you think @jernoble @scottlaw?

Located this issue from

• #2
• #81
• #83
• 6daf2f2
• #122.

The title of the linked Chrome issue https://bugs.chromium.org/p/chromium/issues/detail?id=949570 does not contain the language of this feature request. To avoid confusion, filing this issue for clarity.

Note, it is not immediately obvious that the PictureInPicture window can be resized by user action.

It is apparently also not specified or possible by means of a workaround that have yet succeeded to resize PictureInPicture window programmatically. Persisting a user-resized Picture-in-Picture window requires running the code twice to begin recording at the maximum width or await pictureInPicture.width === screen.width/2 within resize event, then proceed, which is not necessarily possible for each attempt due to the implementation of the resized window, where the output should be met at the first attempt, without awaiting executing requestPictureInPicture() twice or width and height values at resize event initiated by user action.

(Though beyond the scope of this specific issue it should also be possible to resize Picture-in-Picture to fullscreen)

The use case for setting width and height of Picture-in-Picture programmatically, without user action, is recording PictureInPicture window, to utilize the unique playback capabilities of the Picture-in-Picture window.

Video meetings web apps would benefit from automatic Picture-in-Picture behavior when user switches back and forth between web app and other applications/tabs. This is currently not possible with the user gesture requirement in https://wicg.github.io/picture-in-picture/#request-pip (step 6).

After some discussion, we think a declarative behaviour may be the best we can can propose as it allows browsers to correctly implement an “auto-pip on visibility change” feature without opening up a security or annoyance can of worms. Here's what I'm envisioning.

Note: I’m using the autopictureinpicture keyword name. We can use comments to be pick a better name later if needed.

IDL

partial interface HTMLVideoElement {
  attribute boolean autoPictureInPicture;
};

Javascript

const video = document.createElement('video');
video.srcObject = await navigator.mediaDevices.getUserMedia({ video: true });
video.autoPictureInPicture = true; // <- This is where the magic happens.

video.addEventListener('enterpictureinpicture', function(event) {
  // Video entered Picture-in-Picture (potentially automatically)
});
video.addEventListener('leavepictureinpicture', function(event) {
  // Video left Picture-in-Picture (potentially automatically)
});

When autoPictureInPicture is set to true, video will attempt to enter Picture-in-Picture automatically (without user gesture) when page becomes hidden. And existing Picture-in-Picture window will be closed automatically when page becomes visible. We'll use Page Visibility API events for this.

I believe this proposal is simple and concise but there is something to consider.
What if we have two video elements with autoPictureInPicture (or more). Spec could say that the first video element matching document.querySelector('video[autopictureinpicture]') will always get the “auto-pip on visibility change” behaviour.
But what if the video element is not attached to the DOM? In that case, we would have to monitor all autoPictureInPicture values set, potentially managing a stack of video elements allowed to use this feature. Eventually, this would have to be defined in the spec. In the end, there can be only one.

Note that I’ve considered alternatives such as a document.autoPictureInPictureElement attribute that web developers could set. However I’d prefer using a HTML content attribute as this doesn't require web developers to use Javascript for such a simple opt-in thing. And it also pollutes the document element.

What do you think?

Chrome 70, macOS 10.13.6

This example doesn't work for me:

<canvas id="canvas"></canvas>

<button id="pipButton"></button>

<script>
  pipButton.addEventListener('click', function() {
    const video = document.createElement('video');
    video.autoplay = true;
    video.srcObject = canvas.captureStream(60 /* fps */);
    video.requestPictureInPicture();
  });
</script>

I got an error:
Uncaught (in promise) DOMException: Metadata for the video element are not loaded yet.

P.S. If I try the same (video.requestPictureInPicture()) here: https://webrtc.github.io/samples/src/content/capture/canvas-video/
I got an error:
Uncaught (in promise) DOMException: Media Streams are not supported yet.

So, where is truth?

Spec says:

5. OPTIONALLY, if the {{disablePictureInPicture}} attribute is present on |video|, throw an {{InvalidStateError}} and abort these steps.

I think here you want to check the value of the IDL attribute value, no? It would be weird if the disablePictureInPicture conditionally appeared and disappeared.

What is confusing here is that the specs seems to be talking about the HTML attribute itself being present... but what matters more (or what is easier to check) is the value being reflected in the IDL attribute (so maybe just initialize it to false).

When the user agent initiates picture in picture there is no way for the page to get access to the PictureInPictureWindow.

Normally the page would gain access to the PictureInPictureWindow when calling requestPictureInPicture().

Mobile Safari initiates picture-in-picture when app is suspended while playing fullscreen video or the user can press the picture-in-picture button on the video fullscreen window. In these cases, the page will still need access to the PictureInPictureWindow in order to respond to window size changes.

Proposed solutions:
Add a pictureInPictureWindow property to the element or document.

HTML tends to use the past-tense verb "disabled" in names for Boolean attributes (such as <input>'s disabled attribute), rather than the imperative "disable". Consider renaming disablePictureInPicture to pictureInPictureDisabled to align with this.

Requesting Picture-In-Picture should always be triggered by user activation as we don't want websites to spawn PIP windows when you load them.

The spec was designed to be extensible to any element (not only HTMLVideoElement). @beaufortfrancois and I had a look today and it seems that at the API level, the main change would be to have the partial interface HTMLVideoElement be a partial interface Element. disablepictureinpicture makes a bit less sense for Element but could be kept for consistency. We would need some special steps for videos like having metadata and a video track.

Edit: to clarify something, there would be not that many change in functionality and this is something we are still looking into all of this but we believe it could avoid having to create a to do post processing for cases where a website wants to show some UI on the Picture-in-Picture. The idea is still to have the window non interactive so it should not affect security.

We are still looking into what would be required in order to easily allow websites to send random parts of their DOM to Picture-in-Picture but thought we should open a bug in case of @jernoble, @jyavenard or others have an opinion.

3.1.5 says "If one can’t be created, throw a UnknownError and abort these steps."

I'm not sure UnknownError is the right exception to throw. Also, it might be good to give examples of why it can't be triggered. For example, if it is blocked by FeaturePolicy, we should probably do a SecurityError. Maybe check what other features do in that case. Another reason would be if the system blocked the feature for the application. Should the error be the same, I don't know.

• The Abstract section is a bit thin but not really sure what could be added. Though, maybe s/over the desktop/over other window/?
• "Most desktop and mobile OSs have announced or released platform-level support for Picture In Picture, as have many browsers." should probably be more intemporel as the spec could be read in 5 years where "announced" wouldn't make much sense. Something like "is a common platform-level feature" would do.
• The beginning of the API section should be Non-Normative as it's really an introduciton to the API. It might make more sense in the introduction actually.
• We could have an "Example" section with many examples. WDYT?
• I think having a "Concepts" sections with the algorithms would make sense, WDYT? Though, the algorithm are only used at one place at the moment. Though, I guess they could be re-used if we mention the system/UA PIP interaction
• The Integration section is a bit uncommon. It might make sense to move the Feature Policy in a Security Considerations section. In general, I think the rest could move in other places. For example, the "one window" thing could be in the "concepts" section. Probably same thing for Remote Playback?
• nits: I wouldn't use "enter/exit Picture in Picture algorithm" but "enter/exit picture in picture algorithm" :)
• The "notes" are not really notes. You use "might", "must", "recommend". Notes are meant to be non-normative. In other words, one should be able to implement the spec without reading them.

Also, I have a ton of warnings/errors when I try to build the spec locally. Might be interesting to see how you do this as your generated HTML seems to look okay :)

Glad to see this shipped in Chrome! in the last day or so I saw a lot of great reaction from the press and bloggers about it, congrats!

In today's scenarios (scenario 1):

• No document fragment (video element only)
• No custom controls

I do believe no extra CSS effort is needed, but for future expansion, when supporting a document fragment, do we need to change anything?

Future use cases (scenario 2):

Looking at today's web community reaction in blogs & press, I can see a lot of love and willingness to adapt it! are we going to see:

• Map \ Navigation like in Android & Google maps today (very amazing feature)
• Arbitrary document fragment which the proposal stated interest in supporting

In my personal opinion, as a web developer, I will be facing a little confusion in:

• PiP width is 380px (phone media query) but is not phone, I will likely need different UI\UX for phones and PiP.
• PiP might be resized to equal, 50%, of a user screen width and we might need to change sizes and positions between 380px and 620px for example, scanning the resize with JavaScript might add complications.

Suggestion (is it an overkill 😅)?

picture-in-picture media query

Detect and modify the styles of elements in PiP overlay window with a media query

 @media picture-in-picture and (max-width: 160px) {...}

 @media picture-in-picture and (min-width: 560px) {...}

:in-pip pseudo CSS class

Trigger in-pip to change things in the original doc fragment.

.original-doc-fragment:in-pip {
    opacity: 0.3; 
}

The new requestPictureInPicture API has different parameters and a different algorithm so we should consider renaming it.

copyToPictureInPictureWindow was proposed however this has problems if we called it once and then had the window open and we called it again on a different element we should reset the Picture-in-Picture window before we copy the new element.

The spec today states the following with respect to a UA displaying controls that map to MediaSession actions:

The API will have to be used with the MediaSession API for customizing the available controls on the Picture-in-Picture window.

... the user agent MAY expose Picture-in-Picture window controls that change video playback state (e.g., pause).

It makes sense for a play/pause control to be continually shown in a PiP window, however, for the remaining actions (seekbackward, seekforward, previoustrack, nexttrack, and skipad), it seems like there would be value in specifying that UAs which choose to display controls should either hide or disable UI elements that map to actions that do not have a MediaSessionActionHandler associated with them.

Reading the spec, I noticed in section 4.4:

When the size of the Picture-in-Picture window associated with pictureInPictureElement changes, the user agent MUST queue a task to fire an event with the name resize at pictureInPictureElement.

However, from my understanding, the resize event is fired at the PictureInPictureWindow instance, and not pictureInPictureElement?

We should have a way beyond pictureInPictureEnabled for a website to check whether picture in picture is supported. For example, a site may not wish to show UI if the user agent does not support interactive PIP.

The fact that the pipWindow is "always on top" is not visible enough in the spec. Let's fix this.

One potential use case for picture-in-picture is to display one's own local video camera outside of the page window. For example, many web-based videoconferencing applications use a <video> element to display the local video so the user can see what they are sending. This video element is typically styled with a transform such as transform: scaleX(-1) in order to horizontally flip the display of the attached MediaStream. Horizontal flipping is done to achieve a more natural mirror effect.

Due to #81 and #83 styling in picture-in-picture mode is ignored, which means there is no way to achieve the mirror effect. I'm wondering whether it might make sense to allow for an exception to that rule to permit a transform that horizontally flips the video for this use case?

Intent to Migrate: Picture-in-picture

Hi all, cc @mounirlamouri @jernoble as chairs of the Media Working Group. The recently created Media Working Group is ready to adopt Picture-in-Picture as a W3C Recommendation track deliverable.

Note some of the sections below need some love. I let people involved in the development of this proposal chime in as needed!

Working group decision to adopt

See the charter of the Media Working Group

Proposal

Latest Picture-in-picture draft (Git repo)

Summary

An API to allow websites to create a floating video window always on top of other windows so that users may continue consuming media while they interact with other content sites, or applications on their device.

Motivation and Use Cases

Detailed in the explainer

Compatibility Risk

Please characterize how much we might regret standardizing this new feature were we to change or remove it in the future. If you don’t have a github repository, include in this email links to relevant discussions, or documentation about support for the feature on the Web.

Ongoing technical constraints

Implementations may start on mobile but this feature should be supported in all environments.

Some devices may only support playback of one video at a time, and may not be able to render a video overlay on top of another video.

Link to implementation experience and demos

See implementation status

Data

What data do you have available that indicates that this enhancement will affect many users of the Web. Quantify the fraction of websites that are currently using something similar to this feature. Or, if a new feature, characterize the reason that you expect this to be far reaching.

Picture-in-picture has been available on most TV sets for some time now. Several user agents have implemented a similar feature at the chrome level.

Security and Privacy

See responses to security and privacy questionnaire. Also see the Security and Privacy Considerations section in the latest draft

Accessibility

Some envisioned accessibility implications that need to be reviewed further, such as the rendering of a floating video window always on top of other windows Similarly, and lack of support for captioning in the floating video.

Internationalization

A priori no internationalization implication.

Following example code:

// example 1
video.style.display = 'none';
video.requestPictureInPicture();

// example 2
video.style.width = '10000px';
video.style.height = '10000px';
video.requestPictureInPicture();

I guess the expected outcome is that the style of the video element has no influence at all on how the video element is presented in picture-in-picture mode. Should this be clarified in the spec?

I am so excited about the new API after watching the Google IO session. I want to try it. But is there any Chrome version support the new API?

The examples in this repository do not work at all. Not even the simple check for PiP support.

This is how PiP work:

    if (video.webkitSupportsPresentationMode && typeof video.webkitSetPresentationMode === "function") {
        // Toggle PiP when the user clicks the button.
        pipButtonElement.addEventListener("click", function(event) {
            video.webkitSetPresentationMode(video.webkitPresentationMode === "picture-in-picture" ? "inline" : "picture-in-picture");
        });
    } else {
        pipButtonElement.hidden= true;
    }

It is not entirely clear how the picture-in-picture API interacts with the page visibility API when the picture-in-picture video is visible but the browser itself is minimized:

Page visibility API, visibilityState attribute defines (emphasis added):

On getting, the visibilityState attribute the user agent must run the steps to determine the visibility state:

1. Let doc be the Document of the top level browsing context.
2. If the defaultView of doc is null, return hidden.
3. Otherwise, return the VisibilityState value that best matches the visibility state of doc:
   1. Return "visible" if:
      1. The user agent is not minimized and doc is the foreground tab.
      2. The user agent is fully obscured by an accessibility tool, like a magnifier, but a view of the doc is shown.
   2. Return "hidden" if:
      1. The user agent is minimized.
      2. The user agent is not minimized, but doc is on a background tab.
      3. The user agent is to unload doc.
      4. The Operating System lock screen is shown.

From this section it looks like visibilityState should be "hidden" in such a situation.

However, the enum definition says (emphasis added):

visible
The Document is at least partially visible on at least one screen. This is the same condition under which the hidden attribute is set to false.

From this section it looks like visibilityState should be "visible" in such a situation.

It would probably be a good idea to specify the interaction with page visibility explicitly in the picture-in-picture API.

https://wicg.github.io/picture-in-picture/#request-pip requires throwing when the readyState is HAVE_NOTHING. Does that still make sense when the video element has a MediaStream set as srcObject?

Removing this restriction when the srcObject is a MediaStream would (assuming getDisplayMedia is considered a gesture, see crbug) allow calling getDisplayMedia and then pip-ing the resulting stream. I assume that waiting for loadedmetadata or similar and then calling requestpictureinpicture would loose the user activation.

I know this is bikeshedding, but this specification is basically specifying an overlay window (or an always on top, chrome-less popup). "Picture in Picture" is an extremely awkward name (and long, too) for something that is meant to be extensible towards the general overlay.

There's currently no clear explanation of what the value of the autoPictureInPicture attribute means, which makes it difficult to accurately implement compatibly (or document what it does). Please add a definitive explanation of what this attribute means.

I am guessing that it means that in addition to allowing PiP, a value of true for autoPictureInPicture causes PiP mode to automatically engage when the element is presented. If so, however, what if disablePictureInPicture is true? Can it still be autodeployed even if the user can't manually toggle it (I know, it sounds crazy, but worth being clear on this)?

If it's an autodeploy enable/disable toggle, it should be named autoPictureInPictureDisabled (I've filed issue #160 separately to suggest that disablePictureInPicture be changed to pictureInPictureDisabled, to align better with the other attributes throughout HTML that use "disabled" rather than "disable" when naming attributes.

Or does it reflect that PiP is currently in its automatically deployed state, rather than that autodeploy is enabled?