How are you handling output configuration?

[Jacob]

I have had some recent discussions with people about output buffering. I have typically been siding on the side of faster response time. It has come to my attention that this may not be producing the fastest response on the client side.

What settings are you using for:
PHP:
output_buffering = 4096
zlib.output_compression = Off

Apache/IIS:
Page compression enabled

I would like to hear how people are compressing their pages, and what their experience has been.

At this point, I am leaning towards output buffering at some reasonable value (instead of just "on") with zlib output_compression off. I then want to make sure the webserver has an output buffer that is at least the size of the PHP buffer, plus a few k. Then I want to turn on compression at the webserver level.

Jacob

[kpit]

There are a couple of reasons to use output buffering. The first being that you send headers after you send content like set cookie or cache control. But if this is done in the beginning before any content is not nessary to use output buffering.

The second reason is if you are using compression at the PHP level. This is better handled at the Webserver and leave it up to server to handle the compression and the negoiation. The only reason to consider compression is if you have low bandwidth ether on the server side or client side. This is especially useful for Modem users. (Yes there are still areas that can not get broadband yet in the United States. ) It does come a price of more cpu cycles to compress the data. This may not increase the performance but it will reduce the Internet Packets sent and can give the appearance of faster response times. The compression option has pros and cons. You can reduce the delivery time at a cost of more cpu cycles. There are reasons to use the webserver compression vs. the PHP. When you use the web server compression you compress all objects ( ie. javascript and other direct file downloads) not just the php script output.

[stevec]

Current configuration is php: output buffering off but zlib compression on. PHP Zend Optimiser, Apache compression on using mod_deflate.

This is for a UK server with access from US and Hong Kong (the latter needing as much speed/compression going dueot slow down going via the great firewall of china.

I'll have a play on a test setup to see if i can get any improvements using the alternative suggestion.

[stevec]

A quick question - how do you define the deflate buffer size for the deflate module with apache v2? (Going to try the php output buffer of 4096 as per the suggestion).

[kpit]

Here is the info on mod_deflate and the directives. The specific directive to control window size is DeflateWindowSize it is an integer from 1 to 15. The bigger the window the more compression (usually).

[Jacob]

Thanks for all of the feedback. I am definitely interested in hearing how the tests go.

I have found that complete output buffering at the PHP level (output_buffering=on) leads to lowest page response times. It also leads to very high memory usage. If you have a slow network connection to the client (or large pages [cough]) then you will generate the entire response and just sit there.

Max's comments about the extra benefits of Web Server level compression are definitely on target. We have a bunch of javascript that needs to be downloaded.

Jacob

[mycrmspacegunnar]

I can always recommand to use output compression.
Output compression is very valuable especially for dial up users but it will give a very feelable improve performance for all your users.

The HTML pages created by sugar are quite big, with typically over 150KB each.
Such a page will typically compress to something around 20KB.

Often a sugar webpage is created in less than a second on a well configured server.

But a dial up user will need 20 seconds to download the page!
With compression enabled the dialup user will download the same page in less than 3 seconds.

This is a big improvement of the speed.

Even broadband user will typically speed up the page downloading by about 1-2 seconds per page by enabling compression.

Sometimes people fear to enable compresion as they think this will eat up valuable server resources. While compression needs some CPU resources it will help a lot to free server resources. Typically enabling compression can highly increase the overall server performance.

One example to make this more clear:
The overhead of the compression is totally neglectable if you compared it to the time that the server needs to create the page with PHP.
When you run Sugar on a typical Apache instance then Apache will often consume about 6-30MB per page to create it. This *fat* Apache process will hang around and wait until the last bit of the page was delivered to the users webbrowser.

If for example your users only access 5 pages per second then you will have 5 Apache processes hanging around and using 30 to 150 MB of server memory.

If your users are modem users then they will need in average 20 seconds per page to download it. This means that your apache process will need to stay alive all time and they will block valueable server resources during this time.
In other words if you deliver pages to slow dial up users you will need in our example about 100 Apache childs running simultanoiusly to be able to deliver only 5 pages per second. The Apaches will each occupy a connection to the database engine and they will together consume 600 - 3000 MB of server memory !!

With output compression you often compress the pages to something int he range of a 10th - 5th of the original page size. This means you will reduce the number of similtanios running webserver and needed database connections by the same amount.


Another very valuable method for speeding up the delivery to the user is using ETAGs to versionize your dynamic created webpages. ETAG is a method of the HTTP protocol which was designed to enable webbrowser and webservers to identify if a webbroser has the just dynamic created already cached.

By using ETAGs you can effective eleminate the need to download any HTML for pages that you have visited already and that did not change meanwhile.
So if you browser back and forth in sugar typically halve of the pages you are view you have viewed before already. When using ETAG you will not need to download them at all again which of course will make Sugar feel again much much faster.

Cheers
Gunnar

[mycrmspacegunnar]

Thanks for all of the feedback. I am definitely interested in hearing how the tests go.

I have found that complete output buffering at the PHP level (output_buffering=on) leads to lowest page response times. It also leads to very high memory usage. If you have a slow network connection to the client (or large pages [cough]) then you will generate the entire response and just sit there.

Jacob

If you really want to reduce memory consumption than you should not use Apache with mod_php.

The problem with Apache with mod_php is that every Apache process is increased by the big php module. And if you use persistant database connections then even each of the PHP modules will lock up one database connection Even if the Apache process is only delivering a grafik file.

The extra problem of the normal Apache setup is as you already pointed out, is that the fat Apache instance will stay around until the last bit of the webpage was downloaded by the users webbrowser. Whether you use compression in PHP or do it Apache will make no difference here.

A much smarter way of doing this is to use a fastcgi PHP server which is connected to your webserver.

The fastcgi server will handle all the PHP calls.
The fastcgi server will forward to response to the webserver which will do the delivery and the valueable php instance will immediatly be ready to create new dynamic pages - it will not be blocked until the users webbrowser has downloaded the webpage.
The other big advantage is that your webservers instance are small and will not pool huge amounts of unnused database connections.

Using FASTCGI is a smart move to increase server performance.
If you consider to do this and webserver performance is important to you then you could even consider to leave Apache and to use something which provides more performance.

While there are excellent commercial webserver like "Zeus" which is many times faster than Apache. I can recommend to look into using lighttpd http://www.lighttpd.net/
Its free, easy to use and much faster than Apache.


Cheers
Gunnar

[Jacob]

Gunnar,

Thanks for your well thought out and written reply. I have indeed been talking to people about alternative Server engines. I was primarily thinking of offloading the static content from the PHP Web Server, instead of offloading the PHP content from the static Web Server. I am in the middle of a few projects right now, but I will definitely think about this more.

Hopefully we can setup a trial and measure relative stability and performance.

Jacob