Comments (20)
Hi folks! I've been working on a high-level proposal for images along with @adamsilverstein - Im hoping it covers some, if not all of the topics brought up in this thread. That being said we are most definitely looking for community feedback, insight and input. The <picture>
element is a complex beast so we should hammer down as much detail as we can.
Summary
The proposal (attached below) provides high-level insight into the behaviour and possible implementation of the <picture>
element into WordPress. There are some interesting challenges that I've uncovered:
- How do we ensure this functionality is backwards compatible?
- Styling of
<picture>
elements could lead to regression in themes - I've included markup examples as well as cross-browser tests, by far Safari is the weakest link in the chain.
- I plan to add a section that looks at how other plugins handle
<picture>
element support cc @adamsilverstein - I've also included important core functions that could be extended.
- One of the biggest drawbacks, with lots of questions is the ability to serve appropriately sized, art-directed images across device breakpoints - I don't see how we could do this easily at this point, but its something to at least think about.
Proposal
The proposal document can be found here: https://docs.google.com/document/d/1XX9QERfakIbE55oai5OmDzbDoxOtpo1aKAA7b0cUONs/edit#
Aim
Let's keep the conversation going so we can nail down an approach to this work and start development!
from performance.
5% of WordPress sites use the picture
element.
Query
⚠️ This query processes 1.4 TB
WITH picture AS (
SELECT
url
FROM
`httparchive.pages.2022_01_01_mobile`
WHERE
JSON_VALUE(JSON_VALUE(payload, '$._element_count'), '$.picture') IS NOT NULL
), wordpress AS (
SELECT DISTINCT
url
FROM
`httparchive.technologies.2022_01_01_mobile`
WHERE
app = 'WordPress'
), total AS (
SELECT
COUNT(0) AS total
FROM
wordpress
)
SELECT
COUNT(0) AS wp_picture,
(SELECT * FROM total) AS total,
COUNT(0) / (SELECT * FROM total) AS pct_wp_picture
FROM
picture
JOIN
wordpress
USING
(url)
Results
wp_picture total pct_wp_picture
129,755 2,648,818 0.0490
from performance.
We've been using <picture>
tags for many years now on dozens of sites and it has been great. For legacy browsers we used to include a common set of JS to boot up support for HTML5 features in general, including the <picture>
tag, but we haven't included that in several years.
I've never been a fan of the srcset
attribute on <img>
for responsive, so we just stuck to media queries if needed, and it has worked without any issues as far as I remember, Worst case, someone gets the <img>
tag.
For the most part we use this as an upgrade to WebP for users, but we occasionally use it for art direction and then almost always to crop hero images.
I think the hardest leap for people from a styling perspective is that they need to consider the <picture>
tag as basically a wrapper <div>
around an image. So switching from just an <img>
to one wrapped in a <picture>
tag needs to be treated the same. Sometimes it isn't a problem, but if you are in a grid or flex context, the <img>
might not behave how you initially expected, because the container might be getting partially styled instead.
from performance.
Does
<img>
element CSS class/style applies only to<img>
or to other<source>
tags or to<picture>
tag?
As per MDN:
The selected image is then presented in the space occupied by the element.
So the <img>
node remains the placeholder for the image as chosen by the browser from the available sources, meaning CSS should continue to target <img>
and not <picture>
or <source>
?
from performance.
@ddur great questions!
New DOM element node
Good point, this has been raised before. One approach would be an opt-in approach for the progressive enhancement, themes would declare support for 'picture-element' to get the auto-feature for multiple mime types. Alternately, any theme or plugin could call wp_get_attachment_image
with a parameter indicating they want a picture element.
Our main initial use case for this will be images with multiple mime types (for example AVIF with jpeg fallback), however we need to ensure other use cases (for example varying density or art direction) are also covered by the approach we choose.
I'm not sure, this needs more investigation and likely a developer note to show how to properly apply css when using the picture
element.
How about 'img' element and loading=lazy attribute? Does it apply to picture-source elements?
Yes it does, according to this it is sufficient to set the attribute on the contained img
tag.
from performance.
Reading up on the various possible uses for the picture element here: https://dev.opera.com/articles/responsive-images/ I created a demo pages with
picture
elements and linked elements for testing: preview & sourceA few notes:
- Chrome only seems to use responsive images correctly when sizes are provided as part of the media query (first example), the second/third examples use srcset with sizes, specifying the source with the
type
attribute and work in firefox and safari, in chrome dev the responsive image fail to work with this approach, maybe I have something incorrect in my markup? Appreciate any help here!
@adamsilverstein I will take a look at this for you!
- I created jpeg, webp and avif versions of images at various sizes.
- I added a "WebP" tag to the WebP images so they are easy to identify. I don't have an editor that works with AVIF to add the tags there.
from performance.
@LukaszJaro - I think the biggest issue with the <picture>
element in WP at the moment is the fact that there is no UI to enable consistent art-direction. The element itself was part of the new HTML5 spec, but seems to have lost a fair amount of traction lately. Im not too sure why that is, but it could just be because of how complex images especially images that now need to adapt to different devices behave.
The picture element would essentially allow you to create as many crops as you like that would meet different conditions. You could have portrait on mobile and landscape on desktop, without the need to feature detect or show/hide images with CSS. The browser makes the call based on the conditions provided in <source>
- it is very powerful. But again we may also run into a situation where 1 image, could have 10 possible versions (considering the market of devices we're dealing with)
from performance.
On reference to the article - I think its sound. WP already attempts to solve responsive images using srcset
and sizes
- but there is no space to art direct the images on different devices or screen widths. This is because srcset
on an <img />
tag is only responsible for serving the "correctly" sized image based on a condition or width.
from performance.
I've been digging into how images are stored and generated a bit further and will post an update here soon with a proposed path forward.
from performance.
I started working on a POC in a branch, PR incoming.
Some good reading:
- Responsive images - https://web.dev/serve-responsive-images/
- The
picture
element: https://web.dev/learn/design/picture-element/
Some additional details about my findings and the approach:
Use cases for implementing the <picture>
element
The picture element makes sense when the image has more than one mime type, or when the upload type does not match the original type and the user wants to provide a fallback. Picture element could also be used for art directed images.
Use Case 1: uploaded type converted for sub sizes (one mime type, different than original)
Description: User uploads JPEG, sub sized images are WebP. For maximum compatibility, user wants to display WebP in a picture element, falling back to the uploaded JPEG for browsers that don’t support WebP.
Expected output:
<picture>
<source
srcset="{WebP srcset}"
sizes=""
>
<img
alt=""
src="{original image}"
>
</picture>
Use Case 2: two mime types
Description: At some point in the future when a user uploads JPEG images, WordPress will be capable of converting these to AVIF images (AVIF support was added in PHP 8.1). To get the best performance and widest compatibility, the user wants to display a picture element showing first the AVIF format to supporting browsers, then falling back to JPEG images for other browsers.
Expected output:
<picture>
<source
type="image/avif"
srcset="{AVIF srcset}"
sizes=""
>
<img
alt=""
srcset="{JPEG srcset}"
sizes=""
src="{original image}"
>
</picture>
Use Case 3: multiple mime types
Description: In this case the user wants to display a picture element with src sets for AVIF, falling back to WEBP when AVIF isn’t supported, and finally a fallback to JPEG when WEBP isn’t supported.
Expected output:
<picture>
<source
srcset="{AVIF srcset}"
type="image/avif"
sizes=""
>
<source
srcset="{WebP srcset}"
type="image/webp"
sizes=""
>
<img
alt=""
srcset="{JPEG srcset}"
sizes=""
src="{original image}"
>
</picture>
Sub sized image generation
When users upload an image to WordPress, the system generates sub sized images for front end display (a sub sized image is created for each available size smaller than the uploaded image, in the same mime type format as the upload). Data about the original image as well as each sub-sized image created is stored in post meta in an array with the key _wp_attachment_metadata
. This meta data includes an array keyed on sizes
that contains the sub-sized image data, including the file, width, height and mime type. In the future the sub sized images might not be in the same mime format as the original (as is the case when a JPEG is uploaded and WebP is output as the default), or may contain multiple mime type sub sized images, for example AVIF and JPEG images.
The default data for a typical image would look like this after uploading(image_meta truncated):
array (
'width' => 1500,
'height' => 1000,
'file' => '2021/11/road-1.jpg',
'sizes' =>
array (
'medium' =>
array (
'file' => 'road-1-300x200.jpg',
'width' => 300,
'height' => 200,
'mime-type' => 'image/jpeg',
),
'large' =>
array (
'file' => 'road-1-1024x683.jpg',
'width' => 1024,
'height' => 683,
'mime-type' => 'image/jpeg',
),
'thumbnail' =>
array (
'file' => 'road-1-150x150.jpg',
'width' => 150,
'height' => 150,
'mime-type' => 'image/jpeg',
),
'medium_large' =>
array (
'file' => 'road-1-768x512.jpg',
'width' => 768,
'height' => 512,
'mime-type' => 'image/jpeg',
),
'post-thumbnail' =>
array (
'file' => 'road-1-825x510.jpg',
'width' => 825,
'height' => 510,
'mime-type' => 'image/jpeg',
),
),
'image_meta' =>
array ( ),
'original_image' => 'road-1.jpg',
)
Front end image display
WordPress automatically iterates through the post content and adds srcset
attributes for images placed from the media library (images placed by URL don't work currently). It uses a regex to find media-library images, then calls wp_get_attachment_image
to get each image source. wp_get_attachment_image
in turn calls wp_get_attachment_image_srcset
to generate the srcset.
The image source code for a typical image looks like this (line breaks for clarity):
<img
loading="lazy"
width="1024"
height="683"
src="https://wpdev.localhost/wp-content/uploads/2021/12/road-1-1024x683.jpg"
alt=""
class="wp-image-2617"
srcset=
"https://wpdev.localhost/wp-content/uploads/2021/12/road-1-1024x683.jpg 1024w,
https://wpdev.localhost/wp-content/uploads/2021/12/road-1-300x200.jpg 300w,
https://wpdev.localhost/wp-content/uploads/2021/12/road-1-768x512.jpg 768w,
https://wpdev.localhost/wp-content/uploads/2021/12/road-1.jpg 1500w"
sizes="(max-width: 1024px) 100vw, 1024px"
>
Detour: image edits and sub sizes
The wp_calculate_image_srcset
function, includes code to handle images changed by edits in the media library where each edited file name gets an edit hash appended (eg. “-e1640909072390”) to the filename. Sub sized images of the edited image get the same appended hash so they can be matched to the edited image.
For example if you rotate an image using the media library tools, the edit action creates a series of new files with hash extensions, and the post meta ‘sizes’ array is updated to reference the new files. The edited file urls will then be used for srcset generation. Note that the original_image
meta data does not change, and the original image is never edited or removed, WordPress never changes your uploaded image!
Edited image data:
array (
'width' => 2560,
'height' => 1920,
'file' => '2021/12/IMG_1893-1-1-scaled-e1640909072390.jpg',
'sizes' =>
array (
'medium' =>
array (
'file' => 'IMG_1893-1-1-scaled-e1640909072390-300x225.jpg',
'width' => 300,
'height' => 225,
'mime-type' => 'image/jpeg',
),
'large' =>
array (
'file' => 'IMG_1893-1-1-scaled-e1640909072390-1024x768.jpg',
'width' => 1024,
'height' => 768,
'mime-type' => 'image/jpeg',
),
'thumbnail' =>
array (
'file' => 'IMG_1893-1-1-scaled-e1640909072390-150x150.jpg',
'width' => 150,
'height' => 150,
'mime-type' => 'image/jpeg',
),
'medium_large' =>
array (
'file' => 'IMG_1893-1-1-scaled-e1640909072390-768x576.jpg',
'width' => 768,
'height' => 576,
'mime-type' => 'image/jpeg',
),
'1536x1536' =>
array (
'file' => 'IMG_1893-1-1-scaled-e1640909072390-1536x1152.jpg',
'width' => 1536,
'height' => 1152,
'mime-type' => 'image/jpeg',
),
'2048x2048' =>
array (
'file' => 'IMG_1893-1-1-scaled-e1640909072390-2048x1536.jpg',
'width' => 2048,
'height' => 1536,
'mime-type' => 'image/jpeg',
),
),
'image_meta' =>
array ( ),
'original_image' => 'IMG_1893-1-1.jpg',
)
Note that cropping an image directly in the block editor works differently - a new image is created in the media library with the name “-cropped” and cropped dimensions, all sub-sizes are re-generated for that image.
Enabling additional mime types, a cookbook
How can we enable additional mime types in WordPress to support uses cases 2 & 3?
For example, in use case 3, WordPress would create sub sized images in several formats when users upload images (in any supported format). Sub sized images would be created in AVIF, WEBP and JPEG formats.
Proposed approach
- Internally, the sizes array keys can be any string, so when creating additional images beyond the first mime type - for each mime type we can prepend the mime type to the key names, so a WebP medium image would have the key
images/webp-medium
. wp_calculate_image_srcset
would be updated to accept an optional mime type.- When a mime type is not included, the function would work as is, except it would filter out any additional mime type image sizes from the meta data.
- The function
wp_get_attachment_image
would be updated to automatically output apicture
element when alternate mime type sub sized images are found in the images post meta.- Mime type order and availability for picture element support would be filterable on a per-image basis.
- When the original image doesn’t match the sub sized images, it could be used as a fallback
- When multiple mime types are provided, they can be used as
source
elements for thepicture
element with the final type used for the fallback imagesrcset
.
- Code must also consider image edits
- As described above we need to ensure edited images are handled properly.
- Filters should be enabled so developers can adjust each part of the output, especially the source type, srsset and sizes elements. Existing filters should work to enable developer fine tuning, we should validate that and ensure all context is passed (eg mime type).
- Consider whether to require theme support of
html5
eg withcurrent_theme_supports( 'html5' )
For the purposes of the plugin we can test approaches by filter the post content.
from performance.
-
New DOM element node
<picture>
may break theme or custom CSS selectors. How are you going to fix it? -
Does
<img>
element CSS class/style applies only to<img>
or to other<source>
tags or to<picture>
tag?
from performance.
Hey @ddur, thanks for the questions:
New DOM element node
Good point, we probably want to let themes opt into the behavior so they can adjust CSS accordingly. We could make opting in as simple as checking current_theme_supports( 'picture' )
(as suggested on the trac ticket). Maybe with core image blocks we can improve that by adjusting CSS?
Good question that needs more research; I'm guessing you would need to apply to the picture
or both the img
and picture
elements, the element displayed may depend on browser support for the image format or for the picture
element itself.
from performance.
Reading up on the various possible uses for the picture element here: https://dev.opera.com/articles/responsive-images/ I created a demo pages with picture
elements and linked elements for testing: preview & source
A few notes:
- Chrome only seems to use responsive images correctly when sizes are provided as part of the media query (first example), the second/third examples use srcset with sizes, specifying the source with the
type
attribute and work in firefox and safari, in chrome dev the responsive image fail to work with this approach, maybe I have something incorrect in my markup? Appreciate any help here! - I created jpeg, webp and avif versions of images at various sizes.
- I added a "WebP" tag to the WebP images so they are easy to identify. I don't have an editor that works with AVIF to add the tags there.
from performance.
@adamsilverstein is this something you are continuing to work on once back from the break? If so, could we assign the ticket to you please?
from performance.
How about 'img' element and loading=lazy attribute? Does it apply to picture-source elements?
from performance.
@adamsilverstein is this something you are continuing to work on once back from the break? If so, could we assign the ticket to you please?
I have self assigned, however this is a large task and could use several developers working on it, I will think about how we can break out smaller tasks.
from performance.
@adamsilverstein @jjgrainger updating this issue to be a [Type] Overview as it will act as our main "epic" issue, and we will create sub-issues once we have finalised an approach. Does that work for you?
from performance.
In relation to supporting the picture element in core, I wonder how many WordPress sites already use it? I'm sure some plugins or themes must already use the picture
element.
Maybe @rviscomi can help here to build a query similar to the one we worked on for WebP usage by WordPress version [script], this time looking for picture
element use instead. This would let us track adoption once/if we introduce the feature in core.
from performance.
How well does this article still hold up today?
Even here mentions using srcset for resolution/bandwidth cases.
Here's a use real use case I run into and wondering if picture element would help with this:
I work with a marketer who provides me images so I can upload them as a featured images for articles. The single article template has a blown up big version of the featured image while the recent news block or archives page template shows multiple articles with thumbnail sized images. Sometimes they are cut off/cropped on smaller screen sizes. Would using picture element help in these cases?
Also I'm wondering if anyone knows why the picture element is so underused, was it due to IE11? I checked some random sites to see if they use picture and none of them are using it:
https://www.smashingmagazine.com/
https://html.spec.whatwg.org/
https://getbootstrap.com/
from performance.
@LukaszJaro https://www.smashingmagazine.com/ does use the picture element. I see it on the author's pictures.
from performance.
Related Issues (20)
- Enhancing the Scripts API with a loading strategy HOT 3
- Trac 54171: Twenty Twelve: Replace frontend jQuery usage with vanilla JS
- Twenty Fifteen: Replace frontend jQuery usage with vanilla JS
- Adding fetchpriority=”high” to the LCP image in WordPress core HOT 1
- Avoiding to lazy-load LCP / hero images in WordPress core HOT 1
- Improving the calculation of image sizes attributes
- Add frontend rendering metrics in the performance tests
- Optimizing the autoloaded options database query in WordPress core
- Exploring ways to further enhance database query performance
- Creating a plugin checker tool
- Server Timing: run on admin
- Unlikely wp-before-template duration HOT 1
- Prepare 2.5.0 release HOT 3
- Increase PHPStan to rule level to 1 (and beyond) HOT 3
- Add phpstan-no-private to catch calls of seemingly private functions
- Performance issue has_content method in WP_Widget_Media_Gallery
- Skip adding `[Type] Task` PRs to changelogs
- Consider implementing Plugin Release Confirmation approvals HOT 1
- Performance analysis using an old low-powered Android phone
- Implement script loading strategies (async & defer) in core, themes, and plugins
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.