Museums and the audience comments paradox

I was at the Imperial War Museum for an advisory board meeting for the Social Interpretation project recently, and had a chance to reflect on my experiences with previous audience participation projects.  As Claire Ross summarised it, the Social Interpretation project is asking: does applying social media models to collections successfully increase engagement and reach?  And what forms of moderation work in that environment – can the audience be trusted to behave appropriately?

One topic for discussion yesterday was whether the museum should do some 'gardening' on the comments.  Participation rates are relatively high but some of the comments are nonsense ('asdf'), repetitive (thousands of variants of 'Cool' or 'sad') or off-topic ('I like the museum') – a pattern probably common to many museum 'have your say' kiosks.  Gardening could involve 'pruning' out comments that were not directly relevant to the question asked in the interactive, or finding ways to surface the interesting comments.  While there are models available in other sectors (e.g. newspapers), I'm excited by the possibility that the Social Interpretation project might have a chance to address this issue for museums.

A big design challenge for high-traffic 'have your say' interactives is providing a quality experience for the audience who is reading comments – they shouldn't have to wade through screens of repeated, vacuous or rude comments to find the gems – while appropriately respecting the contribution and personal engagement of the person who left the comment.

In the spirit of 'have your say', what do you think the solution might be?  What have you tried (successfully or not) in your own projects, or seen working well elsewhere?

Update: the Social Interpretation have posted I iz in ur xhibition trolling ur comments:

"One of the most discussed issues was about what we have termed ‘gardening comments’ but to put it bluntly it’s more a case of should we be ‘curating the visitor voice’ in order to improve the visitor experience? It’s a difficult question to deal with… 

We are at the stage where we really do want to respect the commenter, but also want to give other readers a high value experience. It’s a question of how we do that, and will it significantly change the project?"

If you found this post, you might also be interested in Notes from 'The Shape of Things: New and emerging technology-enabled models of participation through VGC'.

Update, March 2014: I've just been reading a journal article on 'Normative Influences on Thoughtful Online Participation'. The authors set out to test this hypothesis:

'Individuals exposed to highly thoughtful behavior from others will be more thoughtful in their own online comment contributions than individuals exposed to behavior exhibiting a low degree of thoughtfulness.' 

Thoughtful comments were defined by the number of words, how many seconds it took to write them, and how much of the content was relevant to the issue discussed in the original post. And the results? 'We found significant effects of social norm on all three measures related to participants’ commenting behavior. Relative to the low thoughtfulness condition, participants in the high thoughtfulness condition contributed longer comments, spent more time writing them, and presented more issue-relevant thoughts.' To me, this suggests that it's worth finding ways to highlight the more thoughtful comments (and keeping pulling out those 'asdf' weeds) in an interactive as this may encourage other thoughtful comments in turn.

Reference: Sukumaran, Abhay, Stephanie Vezich, Melanie McHugh, and Clifford Nass. “Normative Influences on Thoughtful Online Participation.” In Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems, 3401–10. Vancouver, BC, Canada: ACM, 2011. http://dl.acm.org/citation.cfm?id=1979450.

Let's push things forward – V&A and British Library beta collections search

The V&A and the British Library have both recently released beta sites for their collections searches.  I'd mentioned the V&A's beta collections search in passing elsewhere, but basically it's great to see such a nicely designed interface – it's already a delight to use and has a simplicity that usually only comes from lots of hard work – and I love that the team were able to publish it as a beta.  Congratulations to all involved!

(I'm thinking about faceted browsing for the Science Museum collections, and it's interesting to see which fields the V&A have included in the 'Explore related objects' panel (example).  I'd be interested to see any usability research on whether users prefer 'inline' links to explore related objects (e.g. in the 'tombstone data' bit to the right of the image) or for the links to appear in a distinct area, as on this site. )

I'm not sure how long it's been live, but the British Library beta catalogue search features a useful 'Refine My Results' panel on the right-hand side of the search results page.

There's also a 'workspace', where items and queries can be saved and managed.  I think there's a unique purpose for users of the BL search that most sites with 'save your items' functions don't have – you can request items directly from your workspace in advance for delivery when next in the library.  My friendly local British Library regular says the ability to save searches between sessions is immensely useful.  You can also export to delicious, Connotea, RefWorks or EndNote, so your data is transportable, though unfortunately when I tested my notes on an item weren't also exported.  I don't have a BL login so I haven't been able to play with their tagging system.

They've included a link to a survey, which is a useful way to get feedback from their users.

Both beta sites are already useful, and I look forward to seeing how they develop.

Get thee to a wiki – the great API challenge in action

Help us work on an informal, lightweight way of devising shared data, API standards for museum and cultural heritage organisations – museum-api.pbwiki.com is open for business.

You could provide examples of APIs you've used or produced, share your experience as a consumer of web services, tell us about your collections.

Commenting on other people's queries and content is an easy way to get started.  I'd particularly love to hear from curators and collections managers – we should be working together to enable greater access to collections.  If you check it out and none of it makes any sense – be brave and say so!  We should be able to explain what we're doing clearly, or we're not doing it right.

Some background: as announced on the nascent museumdev blog, the Science Museum is looking at releasing an API soon – it'll be project-specific to start with, but we're creating it with the intention of using that as an iterative testing and learning process to design an API for wider use. We could re-invent the wheel, but we'd rather make it easy for people to use what they've learnt using other APIs and other museum collections – the easiest way to do that is to work with other museums and developers. The Science Museum's initial public-facing collections API will be used for a 'mashup competition' based on object metadata from our 'cosmos and culture' gallery.

Speaking of museumdev, I started it as somewhere where I could ask questions, point people to discussions, a home for collections of links and stuff in development.  It's also got random technical bits like 'Tip of the Day: saving web.config as Unicode' because I figure I might as well share my mistakes^H^H^H^H^H^H^H^H learning experiences in the hope that someone, somewhere, benefits.

Happy developers + happy museums = happy punters (my JISC dev8D talk)

This is a rough transcript of my lightning talk 'Happy developers, happy museums' at JISC's dev8D 'developer happiness' days last week. The slides are downloadable or embedded below. The reason I'm posting this is because I'd still love to hear comments, ideas, suggestions, particularly from developers outside the museum sector – there's a contact form on my website, or leave a comment here.

"In this talk I want to show you where museums are in terms of data and hear from you on how we can be more useful.

If you're interested in updates I use my blog to [crap on a bit, ahem] talk about development at work, and also to call for comment on various ideas and prototypes. I'm interested in making the architecture and development process transparent, in being responsive to not only traditional museum visitors as end users, but also to developers. If you think of APIs as a UI for developers, we want ours to be both usable and useful.

I really like museums, I've worked in three museums (or families of museums) now over ten years. I think they can do really good things. Museums should be about delight, serendipity and answers that provoke more questions.

A recent book, 'How does one become a scientist? : survey on the birth of a Vocation' states that '60% of scientists over 30 and 40% of scientists under 30 note claim, without prompting, that the Palais de la Découverte [a science museum in Paris] triggered their vocation'.

Museums can really have an impact on how people think about the world, how they think about the possibilities of their lives. I think museums also have a big responsibility – we should be curating collections for current and future audiences, but also trying to provide access to the collections that aren't on display. We should be committed to accessibility, transparency, curation, respecting and enabling expertise.

So today I'm here because we want to share our stuff – we are already – but we want to share better.

We do a lot of audience research and know a lot about some of our users, including our specialist users, but we don't know so much about how people might use our data, it's a relatively new thing for us. We're used to saying 'here are objects in a case, interpretation in label', we're not used to saying 'here's unmediated access, access through the back door'.

Some of the challenges for museums: technology isn't that much of a challenge for us on the whole, except that there are pockets of excellence, people doing amazing things on small budgets with limited resources, but there are also a lot of old-fashioned monolithic project designs with big overheads that take a long time to deliver. Lots of people mean well but don't know what's possible – I want to spread the news about lightweight, more manageable and responsive ways of developing things that make sense and deliver results.

We have a lot of data, but a lot of it's crap. Some of what we have is wrong. Some of it was written 100 years ago, so it doesn't match how we'd describe things now.

We face big institutional challenges. Some curators – (though it does depend on the museum) – fear loss of control, fear intellectual vandalism, that mistakes in user-generated content published on museum sites will cause people to lose trust in museums. We have fears of getting the IT wrong (because for a while we did). Funding and metrics are a big issue – we are paid by how many people come through our door or come to our websites. If we're doing a mashup, how do we measure the usage of that? Are we going to cost our organisations money if we can't measure visits and charge back to the government? [This is particularly an issue for free museums in the UK, an interesting by-product of funding structures.]

Copyright is a huge issue. We might not even own an object that appears in our collections, we might not own the rights to the image of our object, or to the reproductions of an image. We might not have asked for copyright clearance at the time when an object was donated, and the cost of tracing it might be too high, so we can't use that object online. Until we come up with a reliable model that reduces the risk to an institution of saying 'copyright unknown', we're stuck.

The following are some ways I can think of for dealing with these challenges…
Limited resources – we can't build an interface to meet every need for every user, but we can provide the content that they'd use. Some of the semantic web talks here have discussed a 'thin layer' of application over data, and that's kind of where we want to go as well.

Real examples to reduce institutional fear and to provide real examples of working agile projects. [I didn't mean strictly 'agile' methodology but generally projects that deliver early and often and can respond to the changing technical and social environment]

Finding ways for the sector to reward intelligent failure. Some museums will never ever admit to making a mistake. I've heard over the past few days that universities can be the same. Projects that are hyped up suddenly aren't mentioned, and presumably it's failed, but no-one [from the project] ever talks about why so we don't learn from those mistakes. 'Fail faster, succeed sooner'.
I'd like to hear suggestions from you on how we could deal with those challenges.

What are museums known for? Big buildings, full of stuff; experts; we make visitors come to us; we're known for being fun; or for being boring.

Museum websites traditionally appear to be about where we are, when we're open, what's on, is there a cafe on site. Which is useful, but we can do a lot more.

Traditionally we've done pretty exhibition microsites, which are nice – they provide an experience of the exhibition before or after your visit. They're quite marketing-led, they don't necessarily provide an equivalent experience and they don't really let you engage with the content beyond the fact that you're viewing it.

We're doing lots of collections online projects, some of these have ended up being silos – sometimes to the extent if we want to get data out of them, we have to screen-scrape our own data. These sites often aren't as pretty, they don't always have the same design and usability budgets (if any).

I think we should stick to what we're really good at – understanding the data (collections), understanding how to mediate it, how to interpret it, how to select things that are appropriate for publication, and maybe open it up to other people to do the shiny pretty things. [Sounds almost like I'm advocating doing myself out of a job!]

So we have lots of objects, images, lots of metadata; our collections databases also include people, events, dates, places, businesses and organisations, lots of qualified information around things like dates, they're not necessarily simple fields but that means they can convey a lot more meaning. I've included that because people don't always realise we have information beyond objects and object metadata. This slide [11 below] is an example of one of the challenges – this box of objects might not be catalogued as individual instruments, it might just be catalogued as a 'box of stuff', which doesn't help you find the interesting objects in the box. Lots of good stuff is hidden in this way.

We're slowly getting there. We're opening up access. We're using APIs internally to share data between gallery interactives and the web, we're releasing them as data points, we're using them to provide direct access to collections. At the moment it still tends to be quite mediated access, so you're getting a lot of interpretation and a fewer number of objects because of the resources required to create really nice records and the information around them.

'Read access' is relatively easy, 'write access' is harder because that's when we hit those institutional issues around authority, authorship. Some curators are vaguely horrified that they might have to listen to what the public have to say and actually take some of it back into their collections databases. But they also have to understand that they can't know everything about their collections, and there are some specialist users who will know everything there is to know about a particular widget on a particular kind of train. We'd like to capture that knowledge. [London Transport Museum have had a good go at that.]

Some random URLs of cool stuff happening in museums [http://dashboard.imamuseum.org/, http://www.powerhousemuseum.com/collection/database/menu.php, http://www.brooklynmuseum.org/opencollection/collections/, http://objectwiki.sciencemuseum.org.uk/] – it's still very much in small pockets, it's still difficult for museum staff to convince people to take what seems like a leap of faith and try these non-traditional things out.

We're taking our content to where people hang out. We're exploring things like Flickr Commons, asking people to tag and comment. Some museums have been updating collections records with information added by the public as a result. People are geo-tagging photos for us, which means you can do 'then and now' mashups without a big metadata enhancement budget.

I'd like to see an end to silos. We are kinda getting there but there's not a serious commitment to the idea that we need to let things go, that we need to make sure that collections online shareable, that they're interoperable, that they can mesh with other things.

Particularly for an education audience, we want to help researchers help themselves, to help developers help others. What else do we have that people might find useful?

What we can do depends on who you are. I could hope that things like enquiry-based learning, mashups, linked data, semantic web technologies, cross-collections searches, faceted browsing to make complex searches easy would be useful, that the concept of museums as a place where information lives – a happy home for metadata mapped around objects and authority records – are useful for people here but I wouldn't want to put words into your mouths.

There's a lot we can do with the technology, but if we're investing resources we need to make sure that they're useful. I can try things in my own time because it's fun, but if we're going to spend limited resources on interfaces for developers then we need to that it's actually going to help some group of people out there.

The philosophy that I'm working with is 'we've got really cool things, but we can have even cooler things if we can share what we have with everyone else'. "The coolest thing to do with your data will be thought of by someone else". [This quote turns out to be on the event t-shirts, via CRIG!] So that said… any ideas, comments, suggestions?"

And that, thankfully, is where I stopped blathering on. I'll summarise the discussion and post back when I've checked that people are ok with me blogging their comments.

[If the slide show below has a brown face on a black background, it's the right one – slideshare's embed seems to have had a hiccup. If it's not that, try viewing it online directly.]

[My slide images include the Easter Egg museum in Kolomyya, Ukraine and 'Laughter in Odd Places' event at the Museum of London.]

This is a quick dump of some of the text from an interview I did at the event, cos I managed to cover some stuff I didn't quite articulate in my talk:

[On challenges for museums:] We need to change institutional priorities to acknowledge the size of the online audience and the different levels of engagement that are possible with the online experience. Having talked to people here, museums also need to do a bit of a sell job in letting people know that we've changed and we're not just great big imposing buildings full of stuff.

[What are the most exciting developments in the museum sector, online?] For digital collections, going outside the walls of the museum using geo-location to place objects in their original context is amazing. It means you can overlay the streets of the city with past events and lives. Outsourcing curation and negotiating new models of expertise is exciting. Overcoming the fear of the digital surrogate as a competitor for museum visits and understanding that everything we do builds audiences, whether digital or physical.

'The strikethrough is the canonical symbol of the Web'

Below is a quote from Wired's Chris Anderson on museum, curatorial authority and the long tail, from a Washington Post report, 'Smithsonian Click-n-Drags Itself Forward' on Smithsonian 2.0 ('A Gathering to Re-Imagine the Smithsonian in the Digital Age').

The quote really covers two issues – making failures and mistakes in public and leaving them there, and training external volunteers and experts to curate parts of collections, because no one curator can be authoritative on everything in their remit: "in exchange for a slight diminution of the credentialed voice for a small number of things, you would get far more for a lot of things".

I suspect this is a false dichotomy – there's a place for both internal and external expertise. The Science Museum object wiki doesn't mean the rest of the collection catalogue and interpretation has no value or relevance. The challenge lies in presenting organisation and user-contributed content in the same interface – can those boundaries be removed? Is it wise to try? And what about taking external content back into the catalogue?

This isn't a new conversation for museum technologists, but it's a conversation I'd love to have with curators. I've never been sure how the technologists who get really excited by the possibilities of sharing content online in various ways can go about working with curators to find the best way of managing it so that the public, the collections and the curators benefit.

Anyway, onto Chris Anderson:

The discovery of the "long tail" principle has implications for museums because it means there is vast room at the bottom for everything. Which means, Anderson said, that curators need to get over themselves. Their influence will never be the same.

"The Web is messy, and in that messiness comes something new and interesting and really rich," he said. "The strikethrough is the canonical symbol of the Web. It says, 'We blew it, but we are leaving that mistake out there. We're not perfect, but we get better over time.' "

If you think that notion gives indigestion to an organization like the Smithsonian — full of people who have devoted much of their lifetimes to bringing near-perfect luster to some tiny pearl of truth — you would be correct.

The problem is, "the best curators of any given artifact do not work here, and you do not know them," Anderson told the Smithsonian thought leaders. "Not only that, but you can't find them. They can find you, but you can't find them. The only way to find them is to put stuff out there and let them reveal themselves as being an expert."

Take something like, oh, everything the Smithsonian's got on 1950s Cold War aircraft. Put it out there, Anderson suggested, and say, "If you know something about this, tell us." Focus on the those who sound like they have phenomenal expertise, and invest your time and effort into training these volunteers how to curate. "I'll bet that they would be thrilled, and that they would pay their own money to be given the privilege of seeing this stuff up close. It would be their responsibility to do a good job" in authenticating it and explaining it. "It would be the best free labor that you can imagine."

It didn't go down easily among the thought leaders, who have staked their lives' work on authoritativeness, on avoiding strikethroughs. What about the quality and strength of the knowledge we offer? asked one Smithsonian attendee.

You don't get it, Anderson suggested. "There aren't enough of you. Your skills cannot be invested in enough areas to give that quality."

It's like Wikipedia and the Encyclopedia Britannica, Anderson said. Some Wikipedia entries certainly are not as perfectly polished as the Britannica. But "most of the things I'm interested in are not in the Britannica. In exchange for a slight diminution of the credentialed voice for a small number of things, you would get far more for a lot of things. Something is better than nothing." And right now at the Smithsonian, what you get, he said, is "great" or "nothing."

"Is it our job to be smart and be the best? Or is it our job to share knowledge?" Anderson asked.

BBC experimenting with inline links in articles

I noticed the following link when reading a BBC article today:

BBC: We are trialling a new way to allow you to explore background material without leaving the page.
If you turn on inline links, they appear as subtly blue text against the usual grey. Some have icons indicating which site the link relates to (YouTube, Wikipedia), others don't. Links with an icon open the content directly over the article; links without icons open the link in the same window, taking you from the BBC story. Screenshot below:


The 'Read more' links to a page, Story links trial, that says:

For a limited period the BBC News Website is experimenting with clickable links within the body of news stories.

If you click on one of these links, a window will appear containing background material relevant to that word that is highlighted. The links have been carefully chosen by our journalists.

We are doing this trial because we want to see if you enjoy exploring background material presented in this way. It's part of our continuing efforts to provide the best possible experience.

In addition to background material from the BBC News website, we are also displaying content from other sites, including Wikipedia, You Tube and Flickr.

I'd be really interested to know what the results of the trial are, and I hope the BBC share them. I've been thinking about inline links and faceted browsing for collections sites recently, and while the response would presumably vary if the links were only to related content on the same site, it would be useful to know how the two types of links are received.

The story I noticed the link on is also interesting because it shows how content created in a 'social software' way can be (probably wilfully, in this case) misinterpreted:

"Downing Street has been accused of wasting taxpayers' money after making a jokey video in response to a petition for Jeremy Clarkson to be made PM.

A Conservative Party spokesman said: "While the British public is having to tighten its belts, the government is spending taxpayers' money on a completely frivolous project.""

Giant squid dissection via live video

I've been watching the recording of the live stream of the first ever public dissection by Museum scientists of a giant squid.

Congratulations to everyone involved at Museum Victoria, it's a great use of technology and a great approach to openness. The explanations were beautifully clear, and did a great job of contextualising the research, the process and the animal itself.

I love the paparazzi-style photo flashes as they rolled the trolley out onto the main floor.

20% time – an experiment (with some results)

A company called Atlassian have been experimenting with allowing their engineers 20% of their time to work on free or non-core projects (a la Google). They said:

You see, while everyone knows about Google's 20% time and we've heard about all the neat products born from it (Google News, GMail etc) – we've found it extremely difficult to get any hard facts about how it actually works in practice.

So they started with a list of questions they wanted to answer through their experiment, and they've been blogging about it at http://blogs.atlassian.com/developer/20_percent_time/. It makes for interesting reading, and it's great to see some real evidence starting to emerge.

Hat tip: Tech-Ed Collisions.

How to build a web application in four days

There's been a bit of buzz around 'How To Build A Web App in Four Days For $10,000'. Not everything is applicable to the kinds of projects I'd be involved in, but I really liked these points:
  • The best boost you can give you or your team is to provide the time to be creative.
  • You'll come back to your current projects with a new perspective and renewed energy.
  • It will push your team to learn new skills.
  • Simplify the site and app as much as possible. Try launching with just 'Home', 'Help' and 'About'.
  • Make sure to build on a great framework
  • Be technologically agnostic. If your developers are saying it should be built in a certain language and framework and they have solid reasons, trust them and move on.
  • Coordinate how your designers and developers are going to work together.
  • Get your 'Creation Environment' setup correctly. [See the original post for details]