How could they not be?
The Gem Primogenitor was known to be the first primogenitor to commit suicide after bestowing his perfect elemental manipulation to one of his descendants under the presence of most primogenitors.
As Felix recalled, the Light Primogenitor decided to follow in his footsteps on that day.
But, before he did so, he agreed with the other primogenitors to give the same descendant his Light manipulation.
Unfortunately, that descendant ended up dying.
Yet, the worst part, the perfect elemental manipulations didn't return to the primogenitors!
Ever since that day, they were lost forever!
Lady Sphinx's theory literally implied that the crystal hexagon-shaped egg should have a being inside with perfect Gem Manipulation!