I am currently considering what to do regarding a Zope server that I have converted to Xen. To best manage the servers I want to split the Zope instances into different DomU’s based on organisational boundaries. One reason for doing this is so that each sys-admin will only be granted access to the Zope instance that they run so that they can’t accidentally break anyone else’s configuration. Another reason is to give the same benefit in the situation where one sys-admin runs multiple instances, if a sys-admin is asked to do some work by user A and breaks something else running for user A then I think that user A will understand that when you request changes there is a small risk of things going wrong. If a sys-admin is doing work for user A and accidentally breaks something for user B then they won’t expect any great understanding because user B wanted nothing to be touched!
Some people who are involved with the server are hesitant about my ideas because the machine has limited RAM (12G maximum for the server before memory upgrades become unreasonably expensive) and they believe that Zope needs a lot of RAM and will run inefficiently without it.
Currently it seems that every Zope instance has 100M of memory allocated by a parent process running as root (of which 5.5M is resident) and ~500M allocated by a child process running as user “zope” (of which ~250M is resident). So it seems that each DomU would need a minimum of 255M of RAM plus the memory required for Apache and other system services with the ideal being about 600M. This means that I could (in theory at least) have something like 18 DomU’s for running Zope instances with Squid running as a front-end cache for all of them in Dom0.
What I am wondering about is how much memory Zope really needs, could I get better performance out of Zope if I allowed it to use more RAM?
The next issue is regarding Squid. I need to have multiple IP addresses used for the services due to administrative issues (each group wants to have their own IP), having Squid listen on multiple addresses should not be a big deal (but I’ve never set up Squid in a front-end proxy manner so there may be hidden problems). I also need to have some https operations on the same IP addresses. I am considering giving none of the Xen DomU’s public IP addresses and just using Net Filter to DNAT the connections to the right machines (a quick test indicates that if the DomU in question has no publicly visible IP address and routes the packets to the Dom0 then a simple DNAT in the PREROUTING table does the job).
Is there anything else I should be considering when dividing a server for running Zope under Xen?
Is it worth considering a single Apache instance that talks to multiple Zope instances in different DomU’s?
Why use Squid ? Apache mod_proxy allows caching as well and you will benefit from all Apache features: URL rewriting, SSL handling, virtual host, etc.
Vincent: Good point. Some time ago for the same environment we had a discussion about Squid vs mod_proxy and came to the conclusion that Squid was better due to reports of better performance.
It seems to me that mod_proxy will give better logging and easier management. How much performance might we expect to lose by going from Squid to mod_proxy?
Your point about mod_proxy caching SSL is particularly interesting as it means that we can offset some of the loss in performance we expect to get from using mod_proxy instead of Squid.
I have never used Squid as a reverse proxy, so I cannot say anything about performance. It may be a good point depending on the context.
I am not sure what you mean when you say “caching SSL”. What I meaned by SSL handling is that with Apache, you can configure different SSL certificates for different IP, authenticate user with their SSL certificates (instead of doing instead on individual Apaches), restrict available ciphers or protocols, etc.. I suppose that this is not possible with Squid since it cannot act as an SSL terminator.
Vincent: Zope apparently performs some slow and complex operations to return web pages (I know almost nothing about Zope, other people tell me this – please let me know if I’m wrong). Some of the pages will be served via SSL which can not be cached by a regular Squid configuration. If Apache is caching requests then (in theory at least) it could do the SSL negotiation, pass the request to Zope as non-SSL and then cache the result. Of course this would only work for SSL requests for documents that are not specific to the user (not based on a cookie or a form).
I’m learning about Zope myself now and it truly depends on how long things take to return. If you are using ZODB and everyone is pounding on it then things will take a while to return and it maybe better to use a relational database for indexing etc. Here is a good link http://tarekziade.wordpress.com/2007/10/04/4-tips-to-keep-in-mind-about-the-zodb/. It seems the Zope project itself is now focusing on the issue in more detail.