Use your working hand to keep a slight, steady pressure through the drill while you work, which will help the bit stay in ...
If you're trying to drill perfectly perpendicular holes through a piece of wood and finding yourself continually frustrated, try one of these quick hacks. Got an old CD lying around? Of course you do ...