I'm blogging several conversations on twitter around the subject of innovation and experimentation that I thought were worth saving, not least because I'm still thinking about their implications.
To start with, Lynda Kelly (@lyndakelly61) quoted @sebchan at the Hot Science conference on climate change and museums:
'Museums want everything to be slick and polished for mass audience, we lose capacity to be experimental and rapid'
which lead me to tweet:
'does big museum obsession with polish hinder innovation? ('innovation' = keeping up with digital world outside)'.
which lead to a really interesting series of conversations. Erin Blasco responded (over several tweets):
We can't pilot if it's not perfect. … Need to pilot 15 quick/dirty QR codes but we can't put ANY up unless there are 50 & perfectly, expensively designed & impressive. … So basically not allowed to fail and learn = not allowed to pilot = we spend a bunch of $ and fail anyway? … To clarify: it's a cross-dept project. One dept ok with post-it notes & golf pencils. Two others are not. Kinda deadlock.
I think this perfectly illustrates the point and it neatly defines the kind of 'polish' that slows things down – the quality of the user experience with the QR codes would rest with the explanatory text, call to action and the content the user finds at the other end, not the weight and texture of the paper or vinyl they're printed on. Suddenly you've got extra rounds of emails and meetings for those extra layers of sign-off, a work request or contract for design time, plus all the stakeholder engagement that you already, but does that extra investment of time and resources result in a better experiment in audience research?
But kudos to Erin for gettings things this far! (An interesting discussion followed with Erin and @artlust about possible solutions, including holding stakeholder evaluations of the prototypes so they could see how the process worked, and 'making the pilot-ness of it a selling point in the design, letting audiences feel they're part of something special', which made me realise that turning challenges into positives is one of my core design techniques.)
Innovation costs, even my plans to try things cheap/free get scuppered by lack of time. For me less about risk more about resources
Which also rings perfectly true – many potential museum innovators were in this position before the museum funding cuts took hold, so innovating your way out of funding-related crises must be even more difficult now.
On the topic of innovation, Lindsey Green said the 'definite reluctance to pilot and fail impacts innovation'. Rachel Coldicutt had just blogged about 'digital innovation in the arts' in Making Things New, pointing out that the question 'privileges the means of delivery over the thing that’s being delivered', and tweeting that 'innovating a system and innovating art aren't the same thing and perhaps there's more impact from innovating the system'.
If the quest is to, as Rachel problematises in her post, 'use digital technologies to remake the Arts Establishment', then (IMO) it's doomed to failure. You can't introduce new technologies and expect that the people and processes within a cultural organisation will magically upgrade themselves to match. More realistically, people will work around any technology that doesn't suit them (for entirely understandable reasons), and even the best user experience design will fail if it doesn't take account of its context of use. If you want to change the behaviour of people in an organisation, change the metrics they work to. Or, as Rachel says, '[r]ather than change for change’s sake, perhaps we should be identifying required outcomes'. Handily, Bridget McKenzie pointed out that 'The Museums for the Future toolkit includes new eval framework (GEOs = Generic Environmental Outcomes)', so there's hope on the horizon.
The caveats: it's not that I'm against polish, and I think high production values really help our audiences value museum content. But – I think investing in a high level of polish is a waste of resources during prototyping or pilot stages, and a focus on high production values is incompatible with rapid prototyping – 'fail faster' becomes impossible. Usability researchers would also say polished prototypes get less useful feedback because people think the design is set (see also debates around the appearance of wireframes).
It's also worth pointing out my 'scare quotes' around the term 'innovation' above – sadly, things that are regarded as amazing innovations in the museum world are often delayed enough that they're regarded as pretty normal, even expected, by our more digitally-savvy audiences. But that's a whole other conversation…
So, what do you think: does a museum's obsession with polish hinder innovation?
Update, January 2013: Rob Stein has written 'Museum Innovation: Risk, Experimentation and New Ideas', which resonated strongly:
A common pitfall for museums is an unhealthy addiction to monumental undertakings. When massive projects loom with ties to outside support and countless staff hours invested in a single deliverable, it becomes very difficult to admit the possibility of failure. As a result, we shy away from risk, mitigate the probability of embarrassment, and crush innovation in the process.
Goal for Friday: fail faster. :)
I'm glad that this issue is getting a bit more air than my 'off-topic' comment at Hot Science (it wasn't an integrated part of my presentation but more a response to what a speaker earlier in my session had said).
The "obsession with polish" is, in the minds of many internal stakeholders, directly related to their desire to reach that fictional "mass audience" that apparently wants museums to be as shiny and unbreakable as a theme park.
This relationship between imagined audience desire and expensive polish is one that needs to be called out directly. The former can be dispelled with actual audience research.
Avoidance of this 'whole-of-organisation' issue means that agile experimentation will remain at the margins – or be the thing "the web team does'.
I don't see it as "polish" for a mass-audience, I see it as obsessive risk-aversion. This may be the same process under a different lens…
Let's take museums for what they are: mostly publicly funded institutions run almost entirely on the reputation of individuals. This is why they are not theme parks. Nobody gets on their high horse when a theme park spends tons of money on a new rollercoaster.
Innovation requires speed, more so when it comes to technology. Taking more than a year to finish a webapp runs a very real risk that it's out of date before it launches.
Seb calls it polish, I'll call it double-checking. It have to go through so many people to double-check. check the language, check the function, check it cannot offend a solitary soul…
when I said on twitter/tumblr what the source of this obsession was, my thoughts that it's Fear. It takes one journalist to say "waste of money" and that's a hit on a museum workers reputation. That same worker has to apply for funding at some point, possibly with the mythical black mark on them.
It this true or paranoid delusion, i honestly can't tell. I'm reminded of the attacks on The Public in the UK midlands. It had technical problems and the newspapers went mad with accusations of wasting money. Then politicians came out, often photographed frowning and pointing, saying how a school is being shut down or a hospital needs repairs.
And that's it. The Public was buried because nobody was there to defend it. I remember the fury when the staff organised knitting workshops, as if it was the high of decadence.
So you cannot risk the museum's name and you certainly cannot risk the reputation of your boss/co-workers. So get it perfect before we even think about telling anyone about it. Good luck.
Multiple, recursive layers of sign-off is definitely a barrier to speed in museums, but I think the desire for extremely high levels of polish is different.
re your point about journalists – fear of a bad review is definitely a barrier to innovation. I'd hope that transparency about what an innovation was trying to achieve might alleviate some of that risk, but I'm not sure it would because there are always other factors at play in any review.
The issue you're pointing to here is that museums are told to innovate or risk becoming irrelevant; then damned for offending traditional museum-goers with sparkly interactives and participatory projects…
re Seb's point – does anyone know of audience research on desired levels of polish? Ideally observation-based as well as direct questions.
Speed is not always desirable and we shouldn't obsess about speed. Instead it is about making the right decisions.
Plenty of museums with good leadership know how to manage negative reviews and political criticism. (A great example is how Arnold Lehman at the Brooklyn has responded to the ongoing public criticism of their offerings).
To think your stakeholders are only the visitors is naive. The key stakeholders are really your revenue sources – primarily government, sponsors, and in the US, Foundations. If you can turn your local community into a source of sustainability for you either through direct funding or by harnessing political influence, then they are crucial too.
As far as the UK situation goes, it wasn't as if widescale political change wasn't able to be predicted, and likewise the economic climate hasn't suddenly blown in from nowhere. It mightn't be the broader situation you want to be in, but your long term planning should be taking these sorts of things into account – it is only going to get more varied with the impact of climate change too.
In fact, a major driver for building an internal work culture that supports innovation is to deal with the rapid pace of change that is coming.
Innovation is not the product of logical thought, although the result is tied to logical structure~Albert Einstein.
I come to this blog via Twitter and @janefinnis who highlighted an article supporting Agile working within ICT units. I am a great supporter of Agile but my gripe with this article was the criticism of Prince2 as an alternative methodology. This in turn has led to the issues raised by museum technologists, most of which apply to any large organisation.
From my limited view most of the issues raised revolve around finance, a perceived lack of managerial support and a lack of standardisation. Many of these issues could be resolved by management introducing recognised methodologies.
I suggest that firstly development must be separated from ongoing support and maintenance. Support and maintenance guys would look after day to day incidents & problems, change control etc. Elements of ITIL Service Management would be a good methodology to use here (and a support team is a good place to train people). Developers should be encouraged to be innovative but MUST work in a controlled environment. Elements of Prince2 for clearly defined projects and Agile working for staged implementations would be good methodologies to introduce as they are both tried and tested approaches.
In my experience (35 years in systems development in UK, USA & Middle East) I find a streamlined use of Prince2 useful for clear developments to agree Scope (scope creep being the biggest killer of projects), ring-fence finance, allocate resource and control activities. Don’t listen to the critics who say channels get clogged with paper – a cut down version without all the bureaucracy is very manageable. For projects where the end game is not so clear Agile is terrific. It allows for a phased development in an organised manner. ALL interested parties must be fully behind a project however and appreciate that total commitment to team working must be given at all times.
For context for other readers, this was the article that critiqued Prince2: http://rewiredstate.org/blog/2011/04/in-defence-of-agile
And the posts here tagged 'manifesto': http://openobjects.blogspot.com/search/label/manifesto
And there was also a link to the survey on frustrations felt by museum technologists at http://openobjects.blogspot.com/2010/11/survey-results-issues-facing-museum.html
And answering for myself… ITIL is used in some museum IT (i.e. support) departments, but I don't know if that's common.
I think ultimately the Prince2/agile-in-museums debate is really about the misapplication of Prince2 in digital projects where the user experience is a critical part of the project's success. Agile development can work within larger Prince2 work packages, but yes, it really does require buy-in. I'm not convinced that some of the reasons given for lack of buy-in are valid – IMO museums need to use the same tried-and-tested tools as other digital developers.
Finally, while I'm here, this post on innovation in libraries, 'What’s keeping you from inventing the future?' http://www.oclc.org/nextspace/018/1.htm rang a lot of bells for me in terms of hindrances to innovation in museums.
Yes, the article 'What’s keeping you from inventing the future?' http://www.oclc.org/nextspace/018/1.htm is very good.
To me the biggest shift will be to make a significant culture change. Many large organisations support bureaucratic departments with their own rules and processes which are bitterly defended when attacked, but these bureaucracies tend to breed a culture of inertia which is a major obstacle to an innovative future. Culture change must come from strong management at the top though. I like Gina Millsap’s statement:
“The key to fostering innovation is great facilitation skills, and this is not something learned in library school. When you sit down with people and call on the collective wisdom of the organization—draw on the expertise of staff—great ideas emerge. You create a culture of continuous process improvement.”
Go to it Mia.
Slick and polished does not preclude experimentation and/or creativity. These are false opposites. This is more about focus, managing expectations, and being prepared to iterate quickly, rapidly, and professionally. I'd also argue that resources are something to accept and work around pragmatically because you will *always* be resource constrained in some way. Don't use resources as an excuse but as a creative constraint.
So, focus. Look at the resources you currently have, and what's *reasonable* to grow into, and what's a stretch. Okay, so that project you have in mind, make it less complex. Tailor it to the resources that you fundamentally have, and find 2-3 places where you're going to stretch. You will need to learn to say no to ideas, features, and experience. You will be saddened along the way, but the things to which you say no become your roadmap to the future, because this first iteration isn't your last.
Think about agile methodologies which suggest sprints and workflows in which something complete and working is delivered on a regular schedule. Not everything is defined perfectly at the start and the project will evolve based on what you learn during development. Be prepared to change and adapt while being mindful of your original goals and make tradeoffs so you're not fundamentally increasing scope. Stay focused.
The first thing you release to the world should be simple and straightforward and you've spent time polishing it because instead of adding new features, you focused on experience and refining it within the usual resources. You're making sure that you don't leave anything that sucks.
So, because you've been planning for the long-game, you have your roadmap in mind, but you also watch the hell out of what people do with your beautiful stuff. They'll quickly show you the places where it's not beautiful and where you were wrong. And, having it in the public where you have to watch the look of disappointment on people's faces will (and if it doesn't stop doing this work) motivate you to want to make it better. If there's something critical, you burn your life at both ends to make it work.
And, of course, you've been managing expectations with stakeholders and supporters and telegraphing the areas of risk and making it clear, like a business, you're listening to your customers / visitors. Because *nobody* expects that you won't make any mistakes, it's how you respond to them that matters. You set the expectation that you're trying to constantly improve and not that there are these moments of where everything is done and you're going to rest. That. Doesn't. Happen. and you set the expectation of how're you're more deeply engaged with success or at least how you're managing it.
And then you iterate. You can either improve the features that aren't as good as you like, you can work on your roadmap, you add new things in response to visitors, or you delete features that aren't needed. You stay focused on the next few steps.
If anyone thinks that companies like Apple or Disney aren't constantly experimenting, innovating, or trying things out, especially with the public, isn't watching their long-game. Yes, you make sure that you make each release as good as possible (hence removing the suck) but you're always going to keep improving. Remember, there weren't even native apps in the original release of iOS. Even now in iOS 6, Apple has publicly said that they didn't do the new Maps app good enough.
Build, release, evaluate, repeat.
Why does it hinder is the first part of the question, the how to change the equation the root of the matter. I agree with the discussion on culture change and the importance of facilitation. I disagree that it needs to start at strong management at the top, though that certainly helps.
Museums are not monoliths, while there are common factors that stifle a more agile process, each institution is unique in their own stifling. Since at a fundamental level most barriers aren't rational, change is created through the use of emotional intelligence rather than rational argument. Finding those levers, identifying the needs and resistances at a person-by-person basis is the real work. In creating an internal evaluative culture, we make sure to start with projects that have rapid payoff and don't address the most sacred cows immediately. Get confidence, build excitement. Buy-in takes time, and practice runs. Be as concrete as possible.
Seen through the lens of Dan and Chip Heath's book Switch, you have your own road map (a la Bruce) to provide the leadership (as mentioned by Seb), build the motivation for others, including their own individual (not necessarily rational) needs and finally you have to present the path. Clear the way for experimentation to be successful.
In my experience, as long as you keep running experiments, and you learn from them and build on them, the audience doesn't give a damn how polished the first bits are.
Kate, I love your update of Tolstoy, 'each institution is unique in their own stifling'.
Bruce, I've updated the post (following Erin's QR code example) to better define 'polish' in this context cos I don't want to introduce a false dichotomy between experimentation and quality user experiences. But paying a design consultant for a QR code printed beautifully on high-quality paper feels a bit like 'cargo cult' polish – it has all the accoutrements of a polished experience but the effort is misdirected.
'Slick and polished does not preclude experimentation and/or creativity. These are false opposites' – in reality, of course they are, but who said museum cultures had to engage with the same reality as the outside world?
And thank you both for sharing your wisdom about how to move onto solving the underlying issue!
Popping back to share some quotes from 'The Importance of Quick and Dirty':
'But at 37signals, we've become so consumed with producing high-quality products that we've forgotten how to do quick and dirty. You might say that our "just-make-the-damn-thing-as-quickly-as-possible" muscle has atrophied. If you don't use it, you lose it. We've lost it.
It was a great wake-up call. A company gets better at the things it practices. Practice quality, and you get better at quality. But quality takes time, so by working solely on quality, you end up losing something else that's important–speed.
I suspect we're not the only company dealing with this problem. In fact, I bet that obsessing about quality too early in the creative process prevents a lot of good ideas from taking shape. As businesses grow, all sorts of things that once were done on the fly–including creating new products–have a way of becoming bureaucratized. As a result, the wrong sets of pressures are brought to bear.
…In many ways, we're returning to our roots: quick, dirty, scrappy, and impatient up front; quality obsessed, careful, and thoughtful later on.'
The strapline says 'A business needs to be able to move at two speeds: fast and slow' – how many museums believe that of themselves? I'd be really curious to know whether things have changed in the almost exactly two years since I wrote this post.