had over half my health taken out in a few shots then decided to retreat. Retreating triggered an instant Game Over because the game was glitched

IMO most likely what happened here is the flagship destroyed the Earth. If the flagship arrives at Earth and sits there for 3 rounds, you get a game over. The only way to stop it is to beat it in a fight, and finishing a phase will buy some time.

This mechanic is definitely not super obvious at first so I wouldn’t blame you for missing it.