I only know how to do this with BF1942 but it can't be that different.
In BF1942 theres a Video.con file, change the line with FullScreen in it to 0 instead of 1.
Heres the similar folder with BF2: C:\Program Files\EA GAMES\Battlefield 2\mods\bf2
Just search through until you find Video.con.