Windows 2008 R2 X64 needs atleast MOSS 2007 with SP2 otherwise you will get "compatibility error" on running setup.exe.
If you have MOSS 2007 or MOSS 2007 SP1: extract the files if they are not already extracted in folder structures. say MOSS2007
Similary extract the MOSS 2007 SP2 in a different folder say MOSS2007SP2
Look for a subfolder Updates in MOSS 2007. Delete any existing files in update folder. Copy all the files from MOSS2007SP2 into the update folder.
Run setup.exe and it works....