Wikipedia has some interesting things to say on the subject. Though most of it is about religion and philosophy, so not directly relevant when talking about technical solutions, it does mention one interesting thing about the term 'evil' in the context of philosophy:

Plato argued that which we call evil is merely ignorance and that good is that which everyone desires. Benedict de Spinoza said that the difference between good and evil is merely one of personal inclinations: "So everyone, by the highest right of Nature, judges what is good and what is evil, considers his own advantage according to his own temperament..." - Wikipedia

In IT, sometimes when we say something is 'evil', it's about something that is evident to any self-respecting coder (illustrated by the way The Daily WTF's Snippet of the Day reveals things we collectively regard as evil). Other times, one man's evil is another man's ambrosia, like for instance the use of the ActiveRecord pattern by Ruby On Rails.

I think the key is in the dictionary definition:

Anything causing injury or harm - Dictionary.com

Or better explained in the 'Jargon File':

...evil does not imply incompetence or bad design, but rather a set of goals or design criteria fatally incompatible with the speaker's. This usage is more an aesthetic and engineering judgment than a moral one in the mainstream sense. ..."
- The Jargon File

(And while you're at it, for some humor you might try searching on Google code search for " evil hack " which has some good results; and the urban dictionary entry on 'evil' which features as number one result the (joke) mathematical proof why girls are evil).

So I would conclude from this that when using the term 'evil' for a technical solution it means that it 'does harm to your aesthetic and engineering judgment'.

Which matches how I usually apply the term to a technical solution.

When I apply it, it generally means that it is an undesirable solution in my judgment, because it either:

1. Is harmful to the quality and legibility of the code (a lot of Regular Expressions that have grown out of control end up being called 'evil')
   
2. Performs some functionality in a place that is completely unrelated to that functionality (infrastructure code in the middle of business logic...) (related to 1.)
   
3. Abuses the design of something by making it do something it was never designed to do (using Flash or images to display text with a specific font, is in my view evil... though sometimes it may be necessary).
   
4. Has to do strange and/or counter-intuitive things to work around bugs in, or limitations of, an external piece of software.

So when my colleague said that caching is evil, I believe it's mainly because of the fourth reason: browsers have to cache images and we have to cache database results, not because it is an integral part of the system, but because it's faster for the browser to pull up the cache than it is to keep requesting the same image again and again.

But you ARE introducing an extra layer of complexity for what is really, in theory, 'not your (a.k.a. the browser's or the php script's) problem'.

However, as anyone will tell you caching is a very necessary evil.

It doesn't make much sense for the browser to keep requesting the same image again and again and it doesn't make much sense for a PHP script to keep performing the same complex query again and again when the data hasn't changed.

So while it does some minor harm to aesthetic and engineering judgment, the user gets a fast experience, which is really all that matters in the end.