At some point in history, Agda's universe levels were just an inductive type like the naturals, but with different accompanying
BUILTIN declarations so that they could have extra judgmental equalities (mentioned in other answers). So, you could do 'weirder' things than you can do now, like define families that depended on the level, rather than being parametric in the level.
I suspect this doesn't cause any foundational problems. Set theorists don't have any issues indexing their universes by ordinals whose fine structure is observable. Also, I don't think it's impossible to have judgmental associativity/commutativity/idempotence of
⊔ on the naturals, as Agda already has some support for adding additional judgmental equalities like that, and I think they're hoping to extend it further.
However, at the time, it made people uneasy to be able to do the 'weird' stuff mentioned above, it wasn't the intended purpose of the first-class universe levels, and the judgmental equality stuff was not really conceived of. So it definitely made sense at the time to make it a bit less like the natural numbers. And it still probably makes sense, for some of these, and other mentioned reasons.