A new open source compression algorithm

Brotli: A new compression algorithm for faster Internet

Brotli: A new compression algorithm for faster Internet
Image by : 

Opensource.com

Brotli is a new open source compression algorithm designed to enable an Internet that's faster for users.

Modern web pages can often be made up of dozens of megabytes of HTML, CSS, and JavaScript, and that's before accounting for images, videos, or other large file content, which all makes for hefty downloads. Such loads are why pages are transferred in compressed formats; they significantly reduce the time required between a website visitor requesting a web page and that page appearing fully loaded on the screen and ready for use.

While the Brotli algorithm was announced by Google in September 2015, only recently have the majority of web browsers adopted it. The HTTP servers Apache and nginx now offer Brotli compression as an option. Besides Google, other commercial vendors (such as Cloudflare and DreamHost) have begun to deploy support for Brotli as well.

Compared to the good ol' gzip file compression and decompression algorithm, Brotli's open source algorithm shrinks data by an additional 20 to 25%. Brotli accomplishes this superior feat by making use of a dictionary that includes the most common words and syllables in English, Spanish, Chinese, Hindi, Russian, and Arabic, as well as common phrases used in machine readable languages (particularly HTML and JavaScript). This gives Brotli a clear advantage in compression and decompression over other algorithms, like gzip, bzip2, LZMA, etc.

To learn more, Cloudflare's engineers offer more details on how Brotli compares to these other algorithms. They provide an insightful exploration with a fascinating viewpoint on the potential impact that Brotli's improved compression may have on real-life applications. Another valuable take on the current state of Brotli (or one as of not too long ago, at least) is by Sam Saffron, co-founder of Discourse.org. He provides another set of benchmarking measures that demonstrate Brotli's size and speed savings, as well as a supportive opinion on why Brotli is mainly available for HTTPS.

With most browsers having added support for Brotli, here at DreamHost we followed suit and began deploying Brotli on our managed WordPress service, DreamPress, starting in December 2016. Since DreamPress uses nginx for TLS termination, we were able to use the Google-built ngx module in order to successfully build and deploy the Brotli library.

While browser support for Brotli is on a swift incline, there are notable holdouts as of this date. Apple Safari still has no plans for supporting Brotli, and Microsoft has plans only for its Edge browser, leaving legacy Internet Explorer users behind in the dust (that's kicked up by speedier alternatives). Microsoft's IIS server does have a free extension to add Brotli compression on the server side.

To check if a website is offering content compressed with Brotli, first make sure that your browser asks for the content-encoding extension br and the response in the header:

(As a sign-of-our-times side note, the initially proposed extension "bro" was rejected for having too much linguistic baggage attached to how the term is used in American English; in any case, naming it br rather than "bro" saves everyone a byte.)

Testing if a site supports Brotli using the tool provided by KeyCDN.com is arguably easier.

While in reality the advantage in speed that comes from using Brotli may not always be so readily apparent to users, it is for real. At the same time, seeing a new open source compression algorithm being developed at all is a rare thing, and that makes it worth appreciating in and of itself. Since we love open source at DreamHost, and all of the stars happened to align, we went ahead and started offering this option. And it works.

Creative Commons License

Tags

8 Comments

Flaco

But what's its Weissman score?

Vote up!
1
Vote down!
0
Tobz

Ah, but does it offer middle-out?
;-)

Vote up!
1
Vote down!
0
WRD

What about performance? If it attains 25%-30% more compression, but the cost of being twice as slow, it won't be much good.

Vote up!
0
Vote down!
0
Krishna Pandey

what are the best level for brotli from 0 to 11 for best results if compared with gzip

Vote up!
0
Vote down!
0
reed

The articles I linked from the post have benchmarks and more details. The gist of it is: the gains in speed and bandwidth depend a lot on how brotli is used.

Vote up!
0
Vote down!
0
QA

Why do all the links in this article have "?target=_blank" in them? What is that supposed to mean?

Vote up!
0
Vote down!
-1
Alex Sanchez

Hi QA,
target="_blank" instructs your web browser to open the link in a new tab or window.

Vote up!
0
Vote down!
0
anatomasovic

Glad to hear there already are plans to implement it widely. I cheered for the "bro" extension, though. :)

Vote up!
0
Vote down!
0

Comment now