(I Can't Get No) Objectivity

Here comes the judge.

We are not, and we never can be, truly objective judges of our teams or their work — no matter how much they wish we were. But subjective opinion is myopic, even dangerous. It’s worth a look.

Forget about team management and leadership for a minute.

Wait, isn’t that what Ars Pandemonium is about? I know, I know. I’ll get back to it in a bit. First I need to do five hundred words on a couple of sixty-year-old pop songs (my references continue to be cutting edge). 

Bear with me. It’ll make sense. I think.

But before I even get to that, I want to set the scene about how I came to understand a few words whose definitions you already certainly know. Then, we’ll talk about team management and leadership.

Look, the whole thing is going to be an adventure.

---

Like a lot of managers of creative professionals, I came up as…a creative professional. It’s not everyone’s trajectory, but it’s pretty archetypal. Become skilled in a discipline → lead a few projects → manage a team.¹

We all know that sometimes this doesn’t work out. Being a good designer, say, doesn’t mean you’re necessarily good at leading designers, much less managing them.² But sometimes it does work!

In my own case, my trajectory went kind of like this: develop skill in discipline → hire team members who are much better than I am at that discipline → discover that I can provide more value in a role that supports those team members.

To be clear, I don’t intend that as some false modesty or humble-brag. I was pretty good at the work! And it took me a few years to understand where my own real value proposition would be. But I think the rest of my career has been a decent testament that I got this right.

Anyway, when I first started doing the “creative direction” part of this work, I had to come to terms with the fact that this required me to judge the creative endeavors of others. Sitting in judgment of creative work has a whiff of Top Chef or American Idol, and I was dubious of it. That’s not how art is supposed to work.

But I knew enough to know that done well, it’s not some thumbs-up/thumbs-down pageant. Rather, it’s a way of pulling together the most cohesive whole, of gently guiding creatives to produce their best work in service of something larger.

I was never — or tried very hard never to be — a director who held some fanatical vision and forced my team to simply execute toward it. For one thing, as I wrote in Manager Judo, I have long subscribed to a notion of “servant leadership” that means I see my role as much more about supporting the vision of the team rather than imposing my own.

But I was also suspicious of my own “subjective” reactions. In my mind, my job was never to make Andrew’s Favorite Product™ — rather it was to try and help whatever project we were working on be as good as it could be. It had to be right in and of itself. I needed to make qualitative judgments that weren’t specifically subjective.³

A subjective assessment is “in the eye of the beholder”; a qualitative one is not, at least so long as the qualities being discussed, and the metrics by which they are being measured, are well understood. At first, I had a sense that to be good at the work, I needed to be more objective in my critique. As it turns out, I think I was right to take a side-eye to subjectivity, but objectivity turned out to be a useless goal, too.

Of course, in any context there are dozens (hundreds! millions! I don’t know how many!) of dimensions along which judgments can be made. In my experience, I realized that paying attention to four in particular could help focus my own decision making. So I told myself the following:

Ignore “objectivity”; it’s unattainable. Be suspicious of “subjectivity”; your opinion is at best meaningless, at worst perilous. Know when a “quantitative” assessment can yield meaningful judgment. And embrace expert “qualitative” evaluations, since with proper definitions and agreed-upon parameters, these are the real meat of creative decision making. 

Just keep repeating: qualitative =/= subjective.

Now, a key component of my project with Ars Pandemonium is to generalize and anonymize events and observations from my own career to share some insights, for whatever they’re worth. As I thought about the various anecdotes I might use to illustrate this essay, I found it very hard to anonymize them. The reality is that describing the kind of judgmental processes at play here gets either meaninglessly vague or uncomfortably specific very quickly.

So instead I’m going to do something fun.

I have decided to try and illuminate my point using one of the great sectarian battles of our time. Which song is better: “(I Can’t Get No) Satisfaction” or “Jumpin’ Jack Flash?” Indeed, many have been lost on this battlefield, but surely the struggle wasn’t in vain.⁴

You know the songs. They’re broadly quite similar — they’re both uptempo rock records written and performed by the same band around the same period (c. late 1960’s). They’re both short, electric guitar riff-based tracks with strong hooks and singable choruses. Anyone remotely familiar with 20th century popular music would instantly recognize them as related to each other. 

But still, they’re not identical...and they’re dissimilar in some interesting ways. For example, “Satisfaction” is more dynamic, with greater contrast between verse and chorus. And its lyrics are somewhat more ambitious, communicating a restless and frustrated sense of…well, I mean…dissatisfaction. 

In comparison, “Flash” is more consistently hard-driving, and its lyrics, while generally evocative rather than specifically meaningful, are pretty positive (“Well it’s alright now/In fact it's a gas”). So they’re similar enough to allow apples-to-apples, but different enough to meaningfully compare. 

So how can we judge which one is better? We have to agree on some parameters.

For example, it is quantitatively true that “Flash” is three seconds shorter than “Satisfaction.” If we say that in pop music shorter is better, then “Flash” is the better song.

On the other hand, if we posit that the better a song is, the longer the artist will make it, then “Satisfaction” is the better track. (That seems silly…but I’d suggest we make much more consequential decisions using even more arbitrary criteria applied to even less significant differences all the time. So there’s that.) 

If we were to agree that the point of pop music is, by definition, to be popular, then we could argue that sales represent a quantitative way to determine the winner: “Satisfaction,” having spent four weeks at number one compared to “Flash’s” peak at number three, is clearly the better song. 

Or! We could argue that the point of a pop song is to get in, get out, and travel light. It should tell you what it’s going to do, do it, then tell you what it did. Then it should punch you in the face, light up a smoke, and go get a beer. If that’s what makes a better song, then surely the harder-driving, more focused “Flash” is the better song! 

Or, on yet some additional hand, if we posited that all of that above was table stakes, and the better song is the one that smartly compromises on those dimensions in order to slip in a bit of meaning and nuance, to add thoughtfulness — even while remaining easy-to-sing — to the lyric, to add dynamics and contrast, then “Satisfaction” is clearly the better song. (And now I’m imagining Elvis Costello agreeing with the latter argument and The Ramones agreeing with the former…and then punching me in the face and lighting up a smoke.) 

The subjective comparison is easy — and meaningless. It’s just which one do you like better? That’s it. That’s all there is to the subjective comparison. This song is better than that song because I like it more. Judgment rendered. 

If I, as your director, assigned you to create a short, catchy, guitar-riff-driven song with lyrics that energetically expressed a sense of ennui and modern dissociation, and you brought me “Flash,” I would assess it as missing the mark. I might like the song, but that’s a subjective determination that’s mostly irrelevant.⁵

This could go on for a while (and I’m having fun!), but I suspect the point is long made: there is no “objectively right" answer to which song is better. Indeed, there’s not even an “objectively right” set of qualitative dimensions on which to judge. 

Okay. That’s all fine and good for assessing and evaluating creative work. But what does it have to do with leading and managing a team of creative people?

Well, I said that my career trajectory was: develop skill → hire team → find my value in supporting team.

But really, those steps were a lot more parallel, or simultaneous, than that. I came up as a director of creative endeavors at essentially the same time that I came up as a director of creative professionals. Those disciplines are inextricably linked in my heart and mind. 

That is to say, many of the skills and tools I developed to be successful at creative direction (to the extent that I have been) are quite similar to those I developed to be successful at team leadership and management (again, such as I’ve been). Of course, a lot of incompetencies, blindspots, and bad habits from one probably infect the other, too. Ah, well.

In any event, what it means is that as I formed this rubric around subjectivity and objectivity, quantitative and qualitative judgements of the work, I also realized a lot of the same principles apply to creating opportunity for — and giving feedback to —  the folks doing that work.

The most important insight here is that my subjective opinion is irrelevant, just as it is in creative assessment. It’s always there, of course. How could it not be? We’re all human beings! We like some people more than we like others. Some people just “fit” better than others. There could be a million reasons why, or no reason at all! It’s how human interaction works. 

But that should not — must not — be the basis of career opportunity or performance evaluation. It can very easily become the cornerstone of an overly homogenous and exclusionary team culture. Would I rather only work with people I really like? Maybe, I suppose. Do I think that would be healthy, effective, or successful? Indeed not.

Just because subjectivity is perilous, though, doesn’t mean there aren’t qualitative evaluations to be made. Of course there are! 

Giving a designer feedback on a design, say, is not necessarily the same as giving them feedback on their job performance. Those surely intersect. But as anyone who’s done this work can attest, the harshest creative critiques often go to the strongest creative performers, and there’s no contradiction at all in that.

There are plenty of quantitative measures that might be applicable, too, but I do tend to think they’re rarer in people management, and they can be a bit of a slippery slope. Creating incentives for more rather than better can backfire.

Still, if say you have ten new features that need written descriptions, and one writer produces three and another writer produces seven, then — assuming the quality of the work was deemed similar — sure, maybe the larger number deserves more recognition.

Or, it might be valuable to assess the sheer quantity of how many issues that a designer closes get re-opened in their particular queue. Are they really applying the process management at the level they should?

But again, none of that is going to create an objective assessment of an individual, their performance, or their potential. Truly, objectivity is a myth and a chimera when dealing with people, just as it is with the work itself.

With all that said, though, I do think this is a place where approaches to creative direction and directing creatives diverge, at least a bit.

As people-managers, we have a responsibility to make judgments in a way that is equitable, transparent, and importantly, comprehensible. Our folks need to know how they will be judged and then understand that they have been judged that way.⁶ They would love to believe we’re objective. We know we can’t be.

So how else can we think about it? Well, I’d suggest something like “replicable.” Doesn’t really roll off the tongue, but I think it describes the objective (see what I did there?) better: would another manager, understanding the same performance criteria, come to the same conclusion about your performance?

That is, when we have effectively defined the parameters by which something (or indeed, someone) will be judged, and we’ve aligned the judgmental skills of the judges, we can believe that they will converge on the same assessment. If they don’t, there’s a problem.

It doesn’t really mean it’s objectively true, but it’s probably the best proxy we’ll get.

The fact is, to my mind at least, there’s no magic bullet here. Most assessments, evaluations, judgments — call them what you will — are frankly qualitative in nature. Knowing when to apply (and when not to apply) quantitative measurements is also important. Chasing objectivity is pointless, but simply relying upon subjectivity in assessments is pointless, too — and risky.

Continually being conscious of which is which — and correctly and consistently using them — is something I think we all struggle with. (Well, anyway I struggle with it and have seen many others struggle, too. Beyond that I’m extrapolating.) I think it’s worth the attention.

What do you think, dear readers? Have you fruitlessly pursued objectivity in evaluations? Have you been tripped up by the difference between qualitative assessments and subjectivity? Do you think “Satisfaction” is just so much disaffected boomer claptrap? Let’s discuss!

1. I know a certain retired architect who might say: become a skilled architect → lead a team of architects → run an architecture firm → go broke. But architecture is, after all, the most cynical of the creative professions.

2. This situation often leads to The Peter Principle being invoked, though that’s usually leveling criticism at a middle manager that really should be aimed at the organization. Bugs me. I should do an essay on that.

3. I wasn’t befuddled by this: I had enough of a background in design, in visual and sonic arts, to know that this is not contradictory. It’s just hard! And doing it professionally takes learning! To quote the automotive designer Frank Stephenson, “Taste is subjective, but the difference between good design and bad design is not subjective. It’s guided by science and the basic laws of proportion and balance.” That’s not universally applicable, but it’s a general notion I hew pretty closely to.

4. Somebody with more modern sensibilities than mine would probably make this between Tortured Poets Department and Cowboy Carter, say. But I, dear reader, have not actually listened to those yet. So boomer white men it shall be!

5. “Mostly irrelevant” here, because if I’m your boss, it probably doesn’t hurt to bring me something I’ll like, even if it’s not a bullseye — I’m human, after all. Also: what an amazing problem that would be, the piece you’re having to turn away being “Jumpin’ Jack Flash”! Oh, my.

6. I wrote about the mechanics of this in The Checklist Is Not a Checklist.