Speeding up Delivery to Client Side

[Jacob]

Also available on the Wiki....Thanks Clint
http://www.sugarcrm.com/wiki/index.php?title=Performance_Tuning_for_HTTPS

Q: Why does loading images take so long over HTTPS?
A: HTTPS is an encrypted connection. Setting up the connection requires multiple round trips between the web server and the browser.

Q: Can we load images through HTTP instead of https when the connection is HTTPS?
A: We cannot use HTTP for images (to answer your earlier question) becuase the end user will get a security warning saying they are being redirected to a non-secure site.

Q: Why is the system slow the first few times that I click through the application?
A: The reason that the initial pages loads are slow is because the system is loading and caching all kinds of things (mostly images and javascript).

I see the browser requesting hundreds of things every day when I first use the application, is there any way to prevent this?
Depending on how long it has been since you have used the images for any given application or website, the browser will check to see if the images have been modified. Explicitly using the webserver to tell the browser how long an image should be cached can greatly increase the time between checks. For Apache, mod_expires and appropriate rules can quickly tell the system to cache *.png, *.jpg for a week.

Possible remedies for slow images:
* Use Server side cache settings to decrease the frequency of checks to see if the images has been modified.
* Connect over HTTP for the first time.
* Create a page that loads all of the images at once, to just get it out of the way. It should load the generic images and all of the theme specific images (for at least the theme being used). We could create a page that says "Loading Cache..." and gives an incremental progress bar of image downloads.

The section below explains mod_deflate and mod_expires. This was copied from a post by:
stevec
http://www.sugarcrm.com/forums/showthread.php?t=8189

Jacob

Some other things to try....

The compression by enabling zlib compression in php.ini is for php code only, you can also enable compression of everything. For apache, install the mod_deflate module:

http://httpd.apache.org/docs/2.0/mod/mod_deflate.html

Also a big help is getting the browser to cache as much as possible (though this will mean users will need to refresh/clear the cache if you edit code or upgrade). This is really useful as a good part of the download for each page is identical javascript. For apache, install the mod_expires modules:

http://httpd.apache.org/docs/2.0/mod/mod_expires.html

and set the config appropriately:

EG. for apache2, modify the site config file in a manner similar to:




<Directory /var/www/>
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
allow from all
AddOutputFilterByType DEFLATE text/html text/plain text/xml text/javascript application/x-javascript text/css
ExpiresByType text/html "access plus 1 day"
ExpiresByType text/css "access plus 1 day"
ExpiresByType text/javascript "access plus 7 days"
ExpiresByType application/x-javascript "access plus 7 days"
ExpiresByType image/gif "access plus 1 month"
ExpiresByType image/jpg "access plus 1 month"
ExpiresByType image/png "access plus 1 month"
ExpiresByType image/x-icon "access plus 1 month"
</Directory>


Don't forget to restart apache

[mycrmspacegunnar]

Great summery Jacob!


There are two more common reasons why HTTPS connection can be very slow.
If a users experiences a very slow https connection I would recommend to check these two points:

The Webbrowser:
There are some versions of Internet Explorer which under some circumstances will not fully cache images files delivered over https even when supplying them with correct expire tags as your proposed.
This not properly caching of images will of course slow down HTTPS when using such affected IE versions.
If your webserver sets the expire tags for your images correctly and your pages load not only on the first page then
giving another webbrowser a try can help. E.g. Firefox

There are a number of websecure tools like Norton Internet Securety which can alter the the webstream and have very negative effects. For example Norton used as proxy (default) will rewrite all http request and will disable HTTP compression and by this slow down a well configured sugar webserver a lot.


Cheers
Gunnar

[Jacob]

One of the problems that I had encountered was the inefficiency of IE 5 or 6 ( not sure of the exact version) where it would create a brand new HTTPS connection for each server side call. If you checked 80 images, that means 80 new HTTPS connections. If you happened to be a few states (not to mention countries or continents) away from the Web Server, you would feel the pain.

I had not thought about all of those lovely little content protection proxies having an effect. This will be added to our list of things to check if someone is complaining about slow server response times. There isn't an easy way to check Apache logs to see if the content was actually delivered as compressed, is there?

Jacob

[stevec]

Another thing to remember with firefox is that, by default, it does not retained caches files and images that were downloaded via https once the browser is closed. To alter this so it does, open a browser window and enter the url:

about:config

filter or search for the field:

browser.cache.disk_cache_ssl

and double click on it to set it to "true".

[Jacob]

Thank you Steve.

This is a good tip. The one concern that I have is that it would cache all SSL pages, including content that should be protected. Do you know of any way to limit it to just images or restrict the sites that it will cache?

Jacob

[stevec]

I'm afraid not - well, I can't find any granularity in that option.

[Jacob]

Jason suggested a page that loads all of the images in the application. My thought was to take that idea and embed it in the login screen. That would essentially start pre-fetching images you are about to need (with UI feedback) while the user is remembering and typing their password.

Has anybody out there seen something like this in action?

BTW, do not attempt to use pre-fetching of links in a Sugar application. The delete links are not so friendly.

[stevec]

Never tried this but it looks interesting:

http://www.netmechanic.com/news/vol6/css_no18.htm

[Jacob]

Thank you Steve. This is just what I was looking for.

[why do I feel long afternoons of clearing cache ahead]

Jacob

[mycrmspacegunnar]

There isn't an easy way to check Apache logs to see if the content was actually delivered as compressed, is there?

Jacob

Hi Jacob,

yes there is, you can add mod_gzip infromation to the Apache logs.
Apache will understand the following compression information tags to be placed into the logfile definition.

# Available 'mod_gzip' compression information 'notes'...
#
# %{mod_gzip_result}n - A short 'result' message. Could be OK or DECLINED, etc.
# %{mod_gzip_input_size}n - The size ( in bytes ) of the requested object.
# %{mod_gzip_output_size}n - The size ( in bytes ) of the compressed version.
# %{mod_gzip_compression_ration}n - The compression rate achieved.

You can add them to your normal apache logfile declaration.
Your declaration will typically look somewhat like this:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common

Some webserver like lighttpd allow to the webapplication to send extra-headers which will be included in the logfile. This is very handy if you want to add the sugar_user_name to the logfile oder add some extra fields like compression/etagging done by php.

But if you don't need that much detail of information but only want to see whether the html files got compressed then the file size will tell us this quite clearly. :-)
Uncomprssed Sugar pages have typically a size of 100-200 KB and the pages will be compressed 15-30 KB only.
So the size difference should be clearly visible.


Cheers
Gunnar

[Jacob]

Thanks Gunnar.

I have created a Wiki Page (http://www.sugarcrm.com/wiki/index.php?title=HTTP_Compression#Apache_GZip_Compr ession_Information) to capture this information and will use it in testing. I am sure a lot of people will benefit from this knowledge.

Jacob