Monday 24 June 2013

A picture is worth a thousand words

When Sir Tim made a band the first image on the web back in 1992, I do not imagine for a moment he understood the full scope of what was to follow. I also do not imagine Marc Andreessen understood the technical debt he was about to introduce and that fateful day in 1993 when he proposed the img tag allowing inline images.

Many will argue of course that the old adage of my title may not apply to the web, where if it were true, every page view would be like reading a novel! and many of those novels would involve cats or one pixel tall  colour gradients.

Leaving aside the philosophical arguments for a moment it takes web browser author a great deal of effort to quickly and efficiently put those cat pictures in front of your eyeballs.

Navigating

Images are navigated by a user in a selection of ways, including:
    Molly the cat
  • Direct navigation to an image like this cat picture is the original way images were viewed i.e not inline and as a separate document not involving any html. Often this is now handled by constructing a generated web page within the browser with the image inline avoiding the need for explicit image content handling.
  • An inline img tag (ironically it really does take thousands of words to describe) which puts the image within the web page not requiring the user to navigate away from the document being displayed. These tags are processed as the Document Object Model (DOM) is constructed from the html source. When an img tag is encountered a fetch is scheduled for the object and when complete the DOM completion events happen and the rendered page is updated.
  • Imported by a CSS stylesheet.
  • inline element created by a script
Whatever the method, the resulting object is subject to the same caching operations as any content object within a browser. These caching and storage operations are not specific to images however images are one of the most resource intensive objects a browser must regularly deal with (though javascript and stylesheet sources are starting to rival it at times) because they are relatively large and numerous.

Pre render processing

When the image object fetch is completed the browser must process the image for rendering which is where this gets even more complicated. I shall use how this works in NetSurf as I know that browsers internals best, but operation is pretty similar in many browsers.

The content fetch will have performed basic content sniffing to determine the received objects mime type. This is necessary because a great number of servers are misconfigured and a disturbingly large number of images served as png are really jpegs etc. Indeed sometimes you even get files served which are not images!

Upon receipt of enough data to decode the image header, for the detected mime type, the images metadata is extracted. This metadata usually includes things like size and colour depth.

If the img tag in the original source document omitted the width or height of the image the entire document render may have to be reflowed at this point to allow the correct spacing. The reflow process is often unsightly and should be avoided. Additionally at this stage if the image is too large to handle or an unhandled format the object will be replaced with the "broken" image icon.

Often that will be everything that is done with the image, when I added "lazy" image conversion to NetSurf we performed extensive profiling and discovered well over 40% of images on visited pages are simply never viewed by the user but that small (100 pixel on a side) images were almost always displayed.

This odd distribution comes down to how images are used in modern web pages, they broadly fall into two categories of "decoration" and "content" for example all the background gradients and sidebar images etc. are generally small images used as decoration whereas the cat picture above is part of the "content". A user may not scroll down a page to see content but almost always gets to "view" the decoration.

Rendering

Created by Andrea R used under CC Attribution-NonCommercial-ShareAlike 2.0 licence
The exact browser heuristics used differ as to when the render operation is performed but they all have a similar job to perform. When i say render here this may be possibly as an "off screen" view if they are actually on another tab etc. Regardless the image data must be converted from the source data (a PNG, JPEG etc.) into a format suitable for the browsers display plotting routines.

The browser will create a render bitmap in whatever format the plotting routines require (for example the GTK plotters use a Cairo image surface) , use an image library to unpack the source image data (PNG) into the render bitmap (possibly performing transforms such as scaling and rotation) and then use that bitmap to update the pixels on screen.

The most common transform at render time is that of scaling, this can be problematic as not all image libraries have output scaling capabilities which results in having to decode the entire source image and then scaling from that bitmap.

This is especially egregious if the source image is large (perhaps a multi megabyte jpeg) but the width and height are set to produce a thumbnail. The effect is amplified if the user has set the image cache size limit to a small value like 8 Megabytes (yes some users do this apparently their machines have 32MB of RAM and they browse the web)

In addition the image may well require tiling (for background gradients) and quite complex transforms (including rotation) thanks to CSS 3. Add in that javascript can alter the css style and hence the transform and you can imagine quite how complex the renderers can become.

Caching

The keen reader might spot that repeated renderings of the source image (e.g. because window is scrolled or clipped) result in this computationally expensive operation also being repeated. We solve this by interposing a render image cache between the source data and the render bitmaps.

By keeping the data in the preferred format, image rendering performance can be greatly increased. It should be noted that this cache is completely distinct from the source object cache and relates only to the rendered images.

Originally NetSurf used to perform the render conversion for every image as it was received without exception, rather than at render time, resulting in a great deal of unnecessary processing and memory usage. This was originally done for simplicity and optimising for "decoration" images.

The rules for determining what gets cached and for how long are somewhat involved and the majority of the code within the current implementation NetSurf uses is metrics and statistic generation to produce better decisions.

There comes a time at which this cache is no longer sufficient and rendering performance becomes unacceptable. The NetSurf renderer errs on the side of reducing resource usage (clearing the cache) at the expense of increased render times. Other browsers make different compromises based on the expected resources of the user base.

Finally

Hopefully that gives a reasonable overview to the operations a browser performs just to put that cat picture in front of your eyeballs.

And maybe next time your browser is guzzling RAM to plot thousands of images you might have a bit more appreciation to exactly what it is up to.

37 comments:

  1. Thank you! Very insightful article!

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. BookMyEssay has best australian writers who are offering best and most trusted Assignment Writing Help in Liverpool to get your best grades at minimal cost

      Delete
  2. Students and researchers looking for neatly formatted dissertations benefit from the dissertation transcription solutions offered by professional transcription companies.

    ReplyDelete
  3. Thank you for this post and now you can check out these medicines.
    Aurogra 100

    Cenforce 150

    Cenforce 200mg

    ReplyDelete
  4. Thanks a lot for this nice post you can buy medicines online :-Cenforce 150

    Cenforce 100mg

    ReplyDelete
  5. Start taking Cenforce 100 professional for getting stronger and sustainable erection

    ReplyDelete
  6. Everyone wants to enjoy a better sex life with their partner but due to having different types of sexual disorder many people not able to enjoy the moments with their partner. The most common sexual disorder in men that ruins men’s sex life is known as Erectile Dysfunction (ED). In ED men wants to have sex sessions with their partner but due to a lack of a harder and effective erection for a long time these men not able to enjoy their sex life. In that case, to cure the problem in an effective manner you can use Cenforce 150
    medicine. It comes in different strengths that will provide you better results in different conditions.
    https://www.omedication.com/


    ReplyDelete
  7. cenforce urges men to accomplish and keep up a firm erection when going together with mental and physical sexual instigation. Buy now at alledmedicines.
    Buy Now:
    cenforce 25

    ReplyDelete
  8. This comment has been removed by the author.

    ReplyDelete

  9. vBridge Hub is a Business Networking platform that connects Tech Vendors, Service Providers, Enterprises and Investors to accelerate the tech adoption process




    Natural Language Processing

    ReplyDelete
  10. Since you know about what a casual letter is, think about the Informal letter design. Composing an ideal casual letter will expect you to find out about the format of informal letter composing design too. We have shared instances of casual letters for your better agreement. To realize more on the best way to compose casual letters, keep perusing…

    ReplyDelete
  11. India Magazine

    India Today is an Indian week after week news magazine distributed by Living Media India Limited, in distribution since 1975 situated in New Delhi. India Today is likewise the name of its sister-distribution in Hindi. Today is important for the India bunch, additionally established in 1975, which presently incorporates 13 magazines, 3 radio broadcasts, 4 TV channels, 1 paper, an old style music name (Music Today), book distributing, and India's just book club. With the distribution of its 30th Anniversary issue in December 2005, the magazine, which had initiated distribution in 1975 with a dissemination of 5,000 duplicates, has distributed five releases and presently has a flow of over 1.1 million duplicates with a readership of over 5.62 million.

    ReplyDelete
  12. Cenforce 100mg are tablets that used by men to cure Erectile Dysfunction (ED) disorder. Buy cenforce 100 medicines online with the fastest shipping.

    ReplyDelete
  13. This comment has been removed by the author.

    ReplyDelete
  14. स्वस्तिवाचन मंत्र ॐ आनोभद्रा: क्रतवो यन्तु विस्वतो · देवानो यथा सदमिद वृधे · देवानां भद्रा

    ReplyDelete
  15. Iverheal 6mg is an effective medicine recommended by doctors to treat parasitic infections of the human body. The drug can become the best remedy for parasitic infections of the eyes, skin and intestines. Each Iverheal 6 tablet can kill or inhibit the action of parasites, causing different disease in human body. Also, the drug acts very quickly and thus provides instant relief. In addition, the course of the medication is also concise.

    ReplyDelete
  16. cenforce 100mg may be a premium product of warrior Labs., that is AN Indian pharma. the corporate is selling this drug globally with an equivalent name. Meanwhile, the virility drug 100mg black pills is obtainable simply within the USA and European countries at illustrious medical stores, medical provides stores, and on-line pharmacies

    ReplyDelete
  17. Sildenafil Citrate, often known as Cenforce 100mg (sildenafil citrate), can be used to treat sexual dysfunction in men. This tablet helps a man's erection grow harder and last longer by increasing blood flow in the genital.

    Cenforce 100mg

    ReplyDelete
  18. Made from Natural Stevia Extracts which is healthy sugar alternative for people. Allows you to consume sweet foods without the added calories

    Sugar Free Stevia Liquid Drops

    ReplyDelete
  19. Cenforce 100 (sildenafil citrate) known as a Blue pill, Sildenafil Citrate will treat sexual feebleness in guys. This tablet assists raise with blooding flow from the penile to assist a man with getting more enthusiastically and keep an extremely lengthy erection.

    ReplyDelete
  20. kamagra-oral-jelly is a male sexual enhancer that makes the erection more vivacious . Contains the dynamic fixing Sildenafil citrate.

    Sildenafil is mostly accessible in tablet structure. Being a badly designed answer for some individuals, Indian makers have chosen to stay away from pointless issues and have made a gel that contains 100 mg of dynamic fixing.

    ReplyDelete
  21. Cenforce 200 is a rounded oral pill that must be taken with a plain glass of water, and if you consume the drug with any other beverages, it can affect your body.Cenforce 150 medication needs to be swallowed whole as crushing or breaking is not allowed. Cenforce 120 dosage of the drug must be according to your physician’s prescription. Cenforce 200 mg is meant to be consumed 45 minutes before the s*xual intercourse, and you must get involved in the s*xual activities, or else the Cenforce 50 drug remains fruitless. Avoid consuming two dosages of the drug in a single day; moreover, there are many other dosages of the product rather than just Cenforce 200. Cenforce D medication starts with 25 mg and ends with 200 mg, which the highest dosage form of Sildenafil citrate.

    ReplyDelete
  22. Fildena 120 is an effective drug used by men with erectile dysfunction. Fildena 120 mg contains sildenafil citrate. Consult your doctor before taking this medicine. Fildena 150 is an effective medicine which is used in the treatment of erectile dysfunction or male impotence. Fildena 100 is a drug that temporarily treats the problem of erectile dysfunction efficiently without being addictive. Fildena 50 is a high quality drug which is also useful in treating blood flow related problems such as pulmonary arterial hypertension. Fildena Super Active contains an active ingredient sildenafil citrate in a concentration of 120 mg. The chemical is a PDE-5 inhibitor that helps relax the muscles, tissues and blood vessels around the pelvic area of ​​the body. Fildena Chewable widens blood vessels and removes clogged arteries, resulting in a free flow of blood. Sildenafil citrate increases the level of cGMP and allows for proper blood flow to the penis. Fildena professional 100mg When blood fills the penis, it causes the penis to become erect and hard longer than usual. Fildena XXX is a safe and effective medicine. Mild side effects may appear during the first few days of taking the drug. It is important to discuss your current health condition with your doctor before starting treatment. The side effects of Fildena 120 include:

    ReplyDelete
  23. It is what can be described as a rescue treatment. Vidalista 80 Each tablet of this drug contains an explosive dose of Tadalafil. Please note that the generally accepted maximum dose of Tadalafil is 20 mg. Vidalista 60 is extremely high, but has been shown to work in about of impotence cases that have not responded to treatment with lower doses. In case you are not familiar with the ingredient in Vidalista 40 drug, Tadalafil, it is a safe compound approved by the FDA in the treatment of impotence. However, Vidalista black 80 use must be approved by a specialist doctor. First of all let's clarify it, because it is often misunderstood. Vidalista does not increase your libido in any way. It doesn't even have a direct impact on your erection. Extra Super Vidalista simply creates a situation conducive to erection. It is also possible to take this drug and not have an erection for hours if there is no s*xual stimulation. The action of Vidalista 20

    medicine is mainly to create a smooth transmission of blood to any organ that needs it. In impotent men, blood movement is restricted because the arteries are strained and have narrow passages.

    ReplyDelete
  24. Vilitra 60 is a drug prescribed for the treatment of erectile dysfunction in men. This medicine is a composition of generic Vardenafil, which is Vilitra 40 main ingredient known to increase the power of men to live a happy s*x life. The type 5 phosphodiesterase inhibitor class includes vardenafil which acts as an enzyme inhibitor that promotes erection in men. Vilitra 20 medicine is available in solid form tablets and is the safest and most useful medicine for solving the problem of erectile dysfunction. It works for almost all men due to its fast action and longevity. The vardenafil in Super Vilitra is a powerful ingredient for correcting erectile dysfunction in men. It is prescribed by doctors for its fast and long-lasting action.

    ReplyDelete
  25. Kamagra oral jelly men will be able to have erections. When there is an inadequate erection, it is one of the useful dosages Super Kamagra that can be bought online. When men consume the drug, it helps to enter the blood and increase the flow, when the process is carried out, men will acquire the ability to achieve maximum erection power. Kamagra Chewable drug has been found to give the safest result and is consumed by men all over the world without a prescription. There is one of the other active Kamagra 100 ingredients when it comes to erectile dysfunction, so in the case of Kamagra Oral Jelly, the ingredient is known as Sildenafil Citrate. When you don't have enough capacity to stand there in mental stress and problems arise.

    ReplyDelete
  26. There are a lot of merge pdf online Makers around which you can use to change essentially any report over completely to the generally well known Split pdf online Configuration. Most Rotate pdf online Makers share two things practically speaking.

    ReplyDelete
  27. This comment has been removed by the author.

    ReplyDelete
  28. This comment has been removed by the author.

    ReplyDelete
  29. This PNG Compressorhas a powerful interface that is simple to use. Users can upload up to 20 images at a time and select the amount of compression for each one. They can also preview the results of their compressing before downloading

    ReplyDelete
  30. To comprehend the significance of Cenforce 100mg, it's essential to understand erectile dysfunction. We will delve into the causes, risk factors, and the impact of ED on men's physical and emotional well-being. By shedding light on this common condition, we can better appreciate the role that Cenforce 100mg plays in restoring sexual function.
    Read more: Cenforce 120mg

    ReplyDelete
  31. This comment has been removed by the author.

    ReplyDelete
  32. Vidalista 20mg, a generic drug made from Tadalafil, is available. The long-lasting effect of this medication is why it's commonly called the "end-of-the week medicine". Vidalista 20 is the equivalent of Cialis and can be prescribed by a specialist to treat erectile dysfunction. Tadalafil is the main component in this medicine and it's good for increasing blood flow to the male regenerative system. It can fight with long-chain treatments.

    Vidalista 40mg is a safe medicine to treat erectile disorder. Vidalista may cause a variety of reactions. To prevent an overdose, the manufacturers of this drug have created different doses. You can see that the name Vidalista is surrounded by numbers. The number represents the dosage power.

    Visit More:-
    Tadalista | Tadapox

    ReplyDelete
  33. Cenforce 120 has been a game-changer for me. It effectively addresses ED concerns, providing a reliable solution for improved performance. The quick onset of action and sustained results make it a convenient choice. I've experienced increased confidence and satisfaction in my intimate moments. I highly recommend Cenforce 120 to anyone seeking a trustworthy solution for ED.
    Buy Also Other Similar Products: Cenforce | Cenforce 100 | Cenforce 150 | Cenforce 200.

    ReplyDelete