Do a Desktop install and then do apt-get remove ubuntu-desktop; apt-get autoremove and poof, you're almost as lean on install as you would have been installing from the Server disc to start with.