It's true that badly coded Swing can crawl, but for this, you can try another widget API such as the semi-native SWT used by the Eclipse project http://www.eclipse.org, or optimize your code
In my opinion the remaining problems of Java in 2005 is that the desktop Java is quite memory and resource hungry (and it's surprinsingly easy to write bad code in Java, resources wise), even if the garbage collector has been improved a lot. And the most common embedded Java platform (MIDP/CLDC) is a bit too limited, even if the JCP is a fast standardization entity (~ 1 year for a standard). For example, it's still not possible to access an external storage device such as a memory card, which is a huge flaw on todays devices (and JSR 75 is too recent and a bit too confusing to be implemented quickly).