Nathan has published an interesting article at Linux.com about second Ethernet port:
Purchase a new PC or motherboard soon, and the chances are good that it will come with two built-in network interfaces -- either two Ethernet jacks or one Ethernet and one Wi-Fi. Tossing in a second adapter is an inexpensive way for the manufacturer to add another bullet point to the product description -- but what exactly are you supposed to do with it? If you are running Linux, you have several alternatives.
My second Ethernet port usage
What's your second Ethernet port usage?