You will receive a penalty to your damage output with magic when you are wearing a lot of armor, or very heavy armor. The chance of misfiring your spell is also increased. If you are wearing heavy armor, your armored magic skill will go up while casting spells. You can find a right point where you can do lot of damage and you armored magic skill raise with little penalty.
There are many schools of magic, each school with their own spells. The more you cast a spell, the better you will be good at it. Higher skilled spells can travel faster through the air and produce more damage.