Jotun's four younger brothers all suddenly thought of their own families. They could not be blamed, for after being alive for so long, their perception of family and family values was bound to be different from normal people. Yet they could not help but take the Innkeepers words as a warning.
Even by their own standards, they had distanced themselves too much from their families, and for tens of thousands of years began treating them more as employees who worked for them rather than family. Perhaps it was time to reassess their relationship a bit. That did not mean they would begin coddling their families, but that at least they should not be so distant.
To some extent, it was true that their lack of oversight had become a huge problem. If not, then how could Noman have been hunted down by outsiders on his family's own planet with none the wiser?