Should I be concerned if my baby gags?

No.  Gagging is a natural reflex that helps your baby avoid choking.

When you start offering solid foods to your baby, they begin to learn how to chew and swallow them.  Sometimes food sticks to the back of their tongue, or goes down their throat before they are ready to swallow.  This may make them gag or cough to protect themselves from choking on the food.  Babies get better at chewing and swallowing over time.