I seem to recall that the lodestone was so important to him because it fell from the sky and so in that way, he always had some sort of a piece of the palace with him (not really the palace,but you know what I mean).
It might not have been a traumatic experience for him at all to lose/give away the lodestone, if he felt that he didn't need it as much as he used to, because now he was in the palace,traveling amongst the stars anyway?
I tend to think so. He just didn't need it anymore; after he trained as master of the palace, the stars belonged to him and that stone became just a pebble to him. Who knows, maybe he gifted a kind Shuna-like human with it, and, since our lives and memory are short, it got lost for a long time.