When is ugly code actually ugly?

In the second part of Beautiful Code in the Real World, Alberto gives hints how to bring back beauty into legacy code bases. He calls them localized actions, following the theme "Scorn Globally, Act Locally".

While reading this post, I'm wondering how to define "ugly" code?

At the moment I'm refactoring a legacy code base, partially written by myself. It works pretty well actually, the reason for refactoring is that the requirements changed and in order to serve the business use cases it needs to be refactored.

At the time of writing that code, it was actually very clever and as said before, it worked pretty well. Looking at it now, I just think it's just ugly and I have the feeling of refactoring all of it. That's not only the case for the code someone else wrote, but also for my own code.

I'm wondering: if you say something is ugly, it's kinda insulting, right? So if code worked pretty well and served it's business requirements, but it looks strange today, is it okay to call it ugly?