-
Notifications
You must be signed in to change notification settings - Fork 44
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use SSIM as progress without normalization #52
Comments
This example you showed reminds me of a video Pat Meenan sent me a while ago, where the webpage just had the text "Hello World!" on a blank page, right after the firstPaint. Such a webpage loading video's PSI should be very close to firstPaint (plus some delta); and this is the correct behavior to expect in my opinion. firstPaint can be replaced with startRender (or firstVisualChange) since these metrics tend to be very close; just different terms used in different repositories. |
The problem here is that in this case, the text is not right after firstPaint, it's substantially farther out, so PSI in this case being very close to firstPaint (which from the user's point of view is indistinguishable from a blank white page) is not a very good outcome. All of the progress from the user's perspective here is when the first text loads, not from a blank white frame with an imperceptible border being painted. |
A followup to #48, we should follow @pahammad's advice and do what visualmetrics does, using SSIM as the progress once the first paint has occurred and do not normalize.
The normalization causes problems on sites like www.chromestatus.com where a white page has a SSIM of 60 (!!!) with the target. This greatly exaggerates the small differences in SSIM later in the page load since the remaining 40% is remapped to 100%.
PSI is ~2600 under the flawed approach when its realistically more like ~1600.
The text was updated successfully, but these errors were encountered: