I tried drawing directly to the frame buffer, the frame rate is much improved, but its causing quite a lot of other problems. The main one being graphical flickering.
There needs to be a backbuffer after all, since the frame contents are displayed after each write rather than at the GXEndDraw() as I originally thought.