Code Monkey home page Code Monkey logo

device-posture's Introduction

favicon

favicon

Device Posture API

Authors

Abstract

This document specifies an API that allows web applications to request and be notified of changes of the posture of a device.

Goals

The device posture is the physical position in which a device holds which may be derived from sensors in addition to the angle. New types of mobile devices are appearing that have some sort of capabilities that allow them to change their posture. The most common types of devices are the ones that can fold (their screen or around their screen), allowing them to physically alter their form factor. The main interest in knowing the posture of a device is to enable new user experiences with responsive design.

A lot of example of responsive designs can be found here.

Complementary existing proposals:

Before discussing the solution proposal - let's overview existing proposals that are relevant and applicable to the problem space. As matter of principle we should generally avoid creating redundant concepts if we can reuse existing platform APIs and capabilities.

  • Viewport Segments API exposes the concept of Viewport Segments, which represent the regions of the window that reside on separate (adjacent) displays or across a seamless folding screen. Because the fold area could be safe or unsafe region to present content, this API lets developers lay around it. In order to target foldable devices it is very likely developers will use the Device Posture API and the Viewport Segments API to build their experiences. This API is also exposed in CSS over here.

  • Presentation API is solving the problem of a effective use of a secondary screen and likely won't attempt to solve challenges outlined above that are specific to devices where a window can span separate physical displays. This would likely still be a separate problem for foldables.

  • Screen Enumeration API Explainer provides information about the physical screen configuration. Web developers might be able to leverage that on foldables, but would need to do extra effort to correlate that information with window parameters. Some concrete examples on why a special purpose API might be useful in addition to Screen Enumeration:

    • Getting adjacency information about spanning window regions to lay out content in several areas in logical way for a device;
    • Getting inner window dimensions that account for application frame, OS UI elements, etc.
  • Window Placement API Explainer is useful in multi-window scenarios on multiple screen devices, but does not target scenarios in which the hosting application (i.e. browser) has a single window which spans multiple displays. In this case, the developer may not wish to open new windows - just hints to help lay out things properly and take advantage of the physical partitioning of the available layout space.

Additionally, while not a solution in the same sense, a "[css-media-queries] Foldables support and enablement" issue discusses the problem space and outlines some details and touches upon outlined issues.

Proposals

New CSS media query: device-posture

We propose a media query that would resolve to a set of fixed postures. These postures consist of a number of predefined values that each encompass a physical state of the device.

Among the values that the device-posture query can take are:

  • folded (applies to laptop/book postures)
  • continuous (applies to flat, tablet, or even seamless curved displays)

The API is also flexible enough that we can add new postures if new devices come to the market.

Examples

/*using the device in a 'book' posture*/
@media (device-posture: folded) { ... }

New JS object in window.navigator: devicePosture

The Navigator property of Window navigator should host device capabilities and system states.

Proposed Object

DevicePosture : EventTarget {
  readonly attribute DevicePostureType type;
  attribute EventHandler onchange;
}

enum DevicePostureType {
  "folded",
  ...,
}

Demos

https://github.com/foldable-devices/demos

Stakeholder Feedback

Mozilla Position: mozilla/standards-positions#882

WebKit Position: WebKit/standards-positions#328

Chrome Status Entry: https://chromestatus.com/feature/5185813744975872

References & Acknowledgements

Many thanks for valuable feedback and advice from:

'drawing of different postures'

device-posture's People

Contributors

anssiko avatar beaufortfrancois avatar darktears avatar diekus avatar github-actions[bot] avatar himorin avatar juhavainio avatar kenchris avatar lauramorinigo avatar marcoscaceres avatar miketaylr avatar rakuco avatar supo2co avatar torgo avatar xfq avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

device-posture's Issues

Definition for "posture" missing

The spec currently lacks a "posture" definition. Given this term can mean many things, we should explicitly define this term in the context of this specification.

The spec has definitions for "current posture" and "posture mode".

Rename "posture modes" to "posture types"

As the DevicePosture interface has now a member called type, we may want to rename posture modes to "posture types" for consistency.
What do you folks think about it?

interface DevicePosture : EventTarget {
  readonly attribute DevicePostureType type;
  ...
};

Improve fingerprinting of screen fold changes

From blink-dev thread: https://groups.google.com/a/chromium.org/g/blink-dev/c/prHGPxF62i4

"Second, screen fold changes (either via onchange or polling) are an ephemeral fingerprinting vector. Given this API is meant for responsive design, we recommend limiting it to visible browsing contexts. I see section 7.2 does constrain onchange for UX reasons. It should also be listed under “Security and Privacy considerations”. The mitigation should also be applied to other ways to query the property, such as polling. (Perhaps defer all updates to the page’s copy of the state until visible, not just the onchange event, or leave the APIs and CSS queries unavailable to hidden pages altogether.)"

rename spec?

It's been discussed renaming the spec to Screen Fold. Lose the Angle bit.

What should be @media and {not} (screen-fold-posture) behavior.

Considering a CSS block like this :

@media all and (screen-fold-posture) {}

or

@media all and not (screen-fold-posture) {}

What the UA should return according to the following cases?

  • Device does not support posture changes
  • Device support posture changes but in which scenario/posture we should evaluate the MQ to true or false?

Support for single/multi-Screen folds, off-center folds, and multiple folds in a single Screen.

Support for single/multi-Screen folds, off-center folds, and multiple folds in a single Screen.
These questions arise from discussion on w3c/window-management#35 and relate to #37

The proposed shape of exposing a single Fold (angle, orientation, and change events) within a Screen seems to indicate that a foldable screen is exposed as a single Screen interface object. However, the proposal notes relevance of "devices with a single flexible screen (seamless), and devices with two screens (with seam)".

I'd like to better understand how this API might handle:

  1. Folds positioned on a seam between two Screens
  2. Folds positioned off-center within a single Screen, or folds between asymmetric Screens (related to #37)
  3. Multiple folds in a single Screen (e.g. 3+ columns, or even orthogonal/overlapping folds within a Screen?)

Please also note the relevance of the proposed Window Segment Enumeration API
Let's seek compatibility between proposals and support forthcoming screen paradigms! I hope we can discuss plans/compatibility in advance of TPAC 2020 or as a Second Screen CG topic; thanks!

CSS Spanning outdated

In the example 2, the draft mention the following:
@media (device-posture: folded) and (screen-spanning: single-fold-vertical) { ... }

The screen-spanning is outdated and should be replaced with vertical-viewport-segments:1, same for flex: 1 1 env(fold-bottom); since the newly updated CSS viewport segments media query.
Anyway, we received some feedback from developers that it can be confusing because it looks like vertical-viewport-segment (screen-spanning in this case) it's part of the current Device Posture API but it's not. Therefore the other fix would be to remove it.
Thoughts?

Drop ScreenFold.angle from the Screen Fold API?

Some editors and implementers of the Screen Fold API attended the Second Screen WG/CG Virtual meeting to discuss this API. During that meeting that group made the following resolution for DAS WG consideration:

RESOLUTION: Drop ScreenFold.angle from the Screen Fold API first version (for DAS WG consideration)

https://www.w3.org/2021/02/24-webscreens-minutes.html#r01

This recommendation was primarily motivated by the following observations:

  • need to focus on the most valuable use cases first,
  • implementability considerations,
  • privacy and security considerations,
  • currently unresolved performance implications of exposing an angle in a CSS media query

I'd like to solicit feedback on this issue whether there are any concerns in scope-limiting the first version of this spec as proposed.

Hearing no concerns, I'd give the editors a mandate to start crafting a PR to remove the angle related API surface from the spec and align the prose with this change.

Once this spec update is complete, I suggest the group aims to publish an updated Working Draft (maybe by the end of Q1 to have a milestone) to inform the broader community of this development.

Device Posture API to include "Semantic Postures"?

Yesterday Microsoft announced a new device that sets difference "postures" even though the screen itself doesn't really change, fold nor physically change (see image below).

image

This got me thinking into the concept and functionality of the device posture API itself, and its relation to these new types of "semantic" postures, and how we should support them. It is becoming increasingly common to have desktop OSs that react to when a device is being used in conjunction to a keyboard (normal laptop) and when it is being used like a tablet (with touch). Generally these changes correspond to spacing out the UI, and I think it would be really interesting to have something similar for websites.

I want to open discussion on these type of "semantic postures" to be included into the API, since I think more and more we realize that a device posture is a lot more than just applicable to foldable screens and dual screen devices.

Think about a video streaming web that when you browse to it you get the normal traditional website, but when you set your device in a "studio", "playing", "consume", "create" (whatever the name it ends up with, I am interested in the action and posture the device is in) it enhances the layout by maybe making the controls bigger, spacing them out and hiding unnecessary menus.

@media (device-posture: studio) { //adds more padding and margins and hides UI elements }

To elaborate a bit more, I think "continuous" doesn't really make the cut when it comes to device posture as for example, the device listed above, and many other 2-in-1s would all fall in the same pocket, and it would be a missed opportunity to provide layouts with better accessibility and usability.

[admin] Enable auto-publish

@xfq completed renaming main branches of W3C GitHub repositories for this group.

For this repo we've now defaulted to gh-pages.

Do we want to use main for the source and deploy to gh-pages similarly to other spec repos?

Regardless, I think master branch is now stale and can be deleted, @xfq?

@kenchris for preference.

Suggestion: Rename to Device Posture API

After having a chat with a bunch of folks, I am suggesting that we rename the spec to Device Posture API and move angle (animation) use-cases elsewhere.

This would fix #45 so we can move on.

Also we have heard that there is interest in postures that don't solely depend on the angle but also on other things like primary input type, so handles cases like table top:

image

image

Also, it is expected that devices will be release with at least two folds which means the user needs to know where the fold is, so it might make more sense if this was part of Window Segments API.

Requesting Accessibility Considerations section

An accessibility review was requested of the APA as part of our role in performing horizontal review of W3C documents for accessibility concerns. The Device Posture APA was reviewed by APA member Paul Grenier who proposed the following comments that were approved by the group.

  1. Per previous discussions, the group would prefer to see summary accessibility considerations organized under a heading at the same level as Security and Privacy Considerations.
  2. In the Accessibility Considerations section, the group would like to see an extrapolation of 1.3.4 Orientation https://www.w3.org/WAI/WCAG21/Understanding/orientation.html applied to device posture until it can be addressed in those documents. For example, "Content does not restrict its view and operation to a single display orientation https://www.w3.org/TR/screen-orientation/ or single device posture https://www.w3.org/TR/device-posture/#posture-modes, unless a specific display arrangement is essential https://www.w3.org/WAI/WCAG21/Understanding/orientation.html#dfn-essential

Linking to plural forms

Important info

Description of problem

ReSpec doesn't seem to support automatically linking to plural forms.

In https://w3c.github.io/device-posture/#reading-the-posture , the "document" links work, but the "documents" link doesn't seem to work. The error is:

Couldn't match "documents" to anything in the document or in any other document cited in this specification: cssom-view, dom, feature-policy, fetch, html, infra, permissions, screen-orientation, url, webidl. Learn more about this error or see how to cite to resolve the error. Occurred at: 1. (Plugin: "core/xref").

Naming of this spec

Adding fold position to identify orientation instead of identify device model

There are some case scenarios where posture depends on the device orientation. For example, if a device is with the posture "laptop" then the browser doesn't identify/support the same way if it's portrait or landscape. In order to identify the posture, developers should check the device model or rely on media queries. Another case scenario is when browsers have multiwindow or popup mode, therefore the fold position will not always be the center of the screen but the posture is still relevant, same thing would apply for devices with more than one fold.
Should the spec add a fold position to prevent this or just rely on media queries?

Move the posture object out of the Screen interface.

Now that the specification only covers the posture it seems weird to have the main entry point of the API to be in the Screen interface.

I strongly think we should move it to another place like window or window.navigator. The thinking is that there is only a single posture at a given time and it's not dependent on the screen.

What does sound best to you folks?

  • window.navigator.posture

  • window.posture which is basically posture

I strongly think that it we land into a generic object like navigator or window we should rename the entry point from posture to device_posture.

Don't say approximately

We can extend this

The posture values table shows how the posture values are derived depending on the orientation of the fold:

with

The values are approximations and might differ per device. For instance, a device might not yield exactly 180° when laying flat, but instead values ranging from 175° to 185°. Device makers SHOULD make sure that the physical device postures map correctly to the postures defined by this specification.

Some devices might also lack one or more of the postures due to physical constraints or device design, in which case the device SHOULD make sure that all combinations of angles and device orientation (which can be locked by [[SCREEN-ORIENTATION]] and host OS) maps into one of the defined postures.

@anssik

Fetching the posture type should be async?

As I'm working on the implementation of the spec inside Chromium (see WIP that I uploaded in https://chromium-review.googlesource.com/c/chromium/src/+/2985142/) it seems clear to me that navigator.devicePosture.type won't work.

On Android and Windows at the moment the posture is derived from the fold angle. In the future we may or not have a dedicated API to query the posture directly but that's what we have for now. On Android it means that we need to fetch the hinge angle from the OS and that's an async operation. On Windows there isn't any API to fetch the hinge angle but so far the foldable devices shipping expose a custom sensor that is basically exposing the hinge angle as well. Similar to Android this is a async operation that may not return immediately. We need to first ask the OS access to the sensor and then wait the readings. This is extremely similar to the generic sensors btw.

So I suggest we change navigator.devicePosture.type to something else with a Promise that we can resolve when the browser have the first information to be able to compute the posture.

@kenchris

logos are preventing auto-publish

I love the logos, but they are preventing us from auto-publishing the spec. We (W3C) generally only add additional logos if we are working with other another standards organization.

Supporting asymmetric shaped displays

Going off the proposal I'm wondering how the current spec could handle multiple displays that aren't direct mirrors of each other. Such as a slide out display or a displays with different orientation from the parent (eg. the lg wing concept).

Some questions:

  • Can the posture queries be chained to target a specific display?
  • Are viewports returned per 'screen'?
  • Does the spec address scroll behavior to allow independent scrolling windows?

Review media query section and value

Not sure if the format of the media query for the screen-fold-angle documentation is correct. Need to get a reference to the 'angle' value and check if it complies with references. @kenchris

environment variable might be useless

It is hard to know what type to use for the environment variable.

seems to make a lot of sense, and that can be used with something like transform: skew(90deg) but I don't see how to convert that to say pixels for moving animations.

CSS calc can only divide and multiply with integer numbers and I don't see any way to convert deg to an integer or <length> value

Fold Angle fingerprinting concerns and its usefulness

Thread in blink-dev: https://groups.google.com/a/chromium.org/g/blink-dev/c/prHGPxF62i4

"First, we’re concerned about the fold.angle property, as that seems a high entropy fingerprinting surface. The specification mentions this, but leaves it to the implementer and does not touch on the level of precision and anonymization the API is trying to achieve. (The suggestions of lowering resolution and fuzzing are really two strategies for the same thing.)"

"Do you have specific use cases that need the full angle information? The spec has an example of angle-driven animations, but that already seems incompatible with the resolution/fuzzing mitigation it suggests. It seems this could use more exploration into use cases and how to balance them with privacy requirements."

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.