Exactly. When using Windows I have Firefox set to download to the desktop, from which I file things into their target folders or the trash. Otherwise, my desktop is icon free.