This will happen if the desktop mode is incompatible with the game mode.
To run in a window, the desktop must be:
The game cannot run in a window on a small, palletised desktop (16 or 256 color).
To change you settings, right-click on the desktop and select 'properties', or use the Settings/Control Panel from the Start menu and select 'Display properties'. Select the tab marked 'settings', and you should see the options for changing the desktop size, and the number of colors.