OS X Lion's FileVault technology may seem to make encrypted virtual memory redundant, but maintaining encryption even with FileVault 2 enabled is likely a good idea.

Without virtualization, even servers attached to a SAN and running the same operating system must have ‘their’ storage preassigned and managed by administrators for it to be accessible ...

For virtual machine hosting as you describe, I agree with RicDavis, in that the correct answer is as much physical memory as you wish to assign to each guest, plus some extra for the hypervisor.

The processor, RAM, and graphics memory are available without restriction. The same goes for other hardware components such as the printer, webcam, and scanner.