Cultivating elements means increasing the orbs in size.
To do that, one would (carefully) infuse the orbs with mana, lots of it.
With limited mana, tempering both physique and elements wasn't advised, so most chose between the two and stuck with their choice.
This was the most deciding factor between Knights and Mages.
Cultivating the physique leads to increased flexibility, strengthened bones, drastic increase in five senses. In essence, an overall increase in body capabilities.
Whereas…
Cultivating elements leads to stronger elemental attacks for less mana, increased aura capabilities, and increased versatility of element. In essence, increase in overall element usage.
This is why Cavarl - a Master-rank mage - was able to conjure blades of wind.
Whereas the Emperor could only use lightning beams to decimate troops or imbue it in his weapon.
'Others don't take both paths because they can't bear mana requirement' Ezra theorized.
He looked at his arm.