First things first: This is how ingrown hairs happen.
1. Don't pick at them with your fingers. Ever.
It can be so tempting, but the easiest way to make an ingrown hair worse is to pick at it. Picking at ingrown hairs inflames the area, and sometimes drives hair further into its reclusive bat cave (aka your skin). Your hands carry outrageous amounts of bacteria, and exposing an ingrown hair to that can cause further swelling, pain, or even infections. Just look at the stuff that grows on your phone. Gross.
Picking at the skin around an ingrown hair often leads to permanent scarring. The momentary satisfaction of getting that sucker out will not be worth it 20 years from now, trust me. Your skin deserves better, and future you will be grateful.