Dave, why do you think those shifts are redundant?
I'm building with cross-gcc from time to time and it's not faster than eVT, quite opposite in fact.
Anyhow, you could build cross-gcc for CE, but it's quite painful.
Here is some howto, largely from memory.
First you will need both binutils and gcc packages, binutils needed because of cross-assembler and other tools (ar, ld) that will be used during GCC build.
You could build it under both Linux and Cygwin, but it's somewhat easier under Linux.
So you downloaded both gcc & binutils source (that's about 50 Megs),
now go to binutils dir and type the following:
./configure --target=arm-wince-pe --prefix=/gcc-ce
If you're building under linux and want to produce windows-hosted tool, you'll need to add --host=i686-pc (or something like that, perhaps "cygwin" must be added).
Replace gcc-ce with whatever location you want it to be installed.
After configure script is done, type "make install", most likely the build will fail in the GDB directory (always fails for me), then go to each tool's dir and build all needed tools one be one:
cd gas
make install
cd ../ld
make install
cd ../bfd
make install
cd ../binutils
make install
Now go to gcc dir and repeat "./configure --target=arm-wince-pe --prefix=/gcc-ce" "make install"
Under Cygwin you may have to edit some system files, because they lack some endianess defines for ARM target.
You may have to tweak some of the config files to exclude thumb & hardware FPU functions from libgcc build.
etc. etc. but with some luck

you'll get at least some of the tools in the gcc-ce dir - I just repeated these step with gcc 3.0.3 and managed to build C++ compiler and all the tools (I had to edit some stuff in lib1funcs.asm, but that's a hack, this could be fixed in config files).
After that you could link *.o files produced by compiler using Microsoft's linker, i.e. simply add them in the eVT IDE. Linker sometimes complains about some stuff in the object file, but I haver looked into it - it just works anyway.
However as I said above you're limited to C, e.g. you can't use GAPI lib from the GCC-compiled sources because GAPI export are C++ mangled names. Similary it's not possible to link C++ code compiled with GCC to eVC-compiled code (well it's actually possible but that's a lot hand-work).
I think I could send you precompiled binaries but this stuff is rather big.
SC