Usability: the key that unlocks geeky goodness

This is a quick pointer to three posts about some usability work I did for the JISC-funded Pelagios project, and a reflection on the process. Pelagios aims to 'help introduce Linked Open Data goodness into online resources that refer to places in the Ancient World'. The project has already done lots of great work with the various partners to bring lots of different data sources together, but they wanted to find out whether the various visualisations (particularly the graph explorer) let users discover the full potential of the linked data sets.

I posted on the project blog about how I worked out a testing plan to encourage user-centred design and set up the usability sessions in Evaluating Pelagios' usability, set out how a test session runs (with sample scripts and tasks) in Evaluating usability: what happens in a user testing session? and finally I posted some early Pelagios usability testing results. The results are from a very small sample of potential users but they were consistent in the issues and positive results uncovered.

The wider lesson for LOD-LAM (linked open data in library, archives, museums) projects is that user testing (and/or a strong user-centred design process) helps general audiences (including subject specialists) appreciate the full potential of a technically-led project – without thoughtful design, the results of all those hours of code may go unloved by the people they were written for. In other words, user experience design is the key that unlocks the geeky goodness that drives these projects. It's old news, but the joy of user testing is that it reminds you of what's really important…

'Shownar: reflecting online buzz around BBC programmes' [read: museum objects]

Call me mildly obsessive (sad, even), but I got really excited when I read this and mentally replaced 'BBC programme' with 'museum object'. From the BBC Internet Blog:

Today sees the launch of Shownar; a new prototype from BBC Vision which aims
to track online buzz around BBC TV and radio programmes and reflect it back in
useful and interesting ways, aiding programme discovery and providing onward
journeys to discussion about those programmes on the wider web.

Shownar aims to track the wealth of activity that takes place around BBC progammes online and work out which are currently gaining the most attention.

So, how does it work? In the first instance, we decided to focus on tracking in-bound links to programme-related pages on bbc.co.uk, so we could be confident that the discussions were actually about a BBC programme … We took a look at a range of possible suppliers, and for this initial prototype chose data provided by Yahoo! Search BOSS, Nielson Online's BlogPulse (which indexes over 100 million blogs), and Twingly (which searches microblogging services like Twitter, Jaiku and Identi.ca for links, even when they are shortened using URL shortening services such as TinyURL and bit.ly). We are also ingesting data from LiveStats, the BBC's own real-time indicator of traffic. Once ingested, this data is processed according to a specially created algorithm to calculate the 'buzz measure' for every BBC programme – more detail on the algorithm can be found on Shownar's Technical information page.

The post discusses some of the interfaces and benefits – I think the possibilities are pretty endless, and will be exploring how it might enhance the discoverability of and harness conversations about the Science Museum's online collections over the year.

Hat tip: @giv_p

Quick and light solutions at 'UK Museums on the Web Conference 2008'

These are my notes from session 4, 'Quick and light solutions', of the UK Museums on the Web Conference 2008. In the interests of getting my notes up quickly I'm putting them up pretty much 'as is', so they're still rough around the edges. There are quite a few sections below which need to be updated when the presentations or photos of slides go online. [These notes would have been up a lot sooner if my laptop hadn't finally given up the ghost over the weekend.]

Frankie Roberto, 'The guerrilla approach to aggregating online collections'
He doesn't have slides, he's presenting using Firefox 3. [You can also read Frankie's post about his presentation on his blog.]

His projects came out of last year's mashed museum day, where the lack of re-usable cultural heritage data online was a real issue. Talk in the pub turned to 'the dark side' of obtaining data – screen scraping was one idea. Then the idea of FoI requests came up, and Frankie ended up sending Freedom of Information requests to national museums in any electronic format with some kind of structure.

He's not showing site he presented at Montreal, it should be online soon and he'll release the code.

Frankie demonstrated the Science Museum object wiki.

[I found 'how it works' as focus of the object text on the Science Museum wiki a really interesting way of writing object descriptions, it could work well for other projects.]

He has concerns about big top down projects so he's suggesting five small or niche projects. He asked himself, how do people relate to objects?
1. Lots of people say, "I've got one of these" so: ivegotoneofthose.com – put objects up, people can hit button to say 'I have one of those'. The raw numbers could be interesting.
[I suggested this for Exploring 20th Century London at one point, but with a bit more user-generated content so that people could upload photos of their object at home or stories about how they got it, etc. I suppose ivegotoneofthose.com could be built so that it also lets people add content about their particular thing, then ideally that could be pulled back into and displayed on a museum site like Exploring. Would ivegotoneofthose.com sit on top of a federated collections search or would it have its own object list?]
2. Looking at TheyWorkForYou.com, he suggests: TheyCollectForYou.com – scan acquisition forms, publish feeds of which curators have bought what objects. [Bringing transparency to the acquisition process?]
3. Looking at howstuffworks.com, what about howstuffworked.com?
4. 'what should we collect next?' – opening up discourse on purchasing. Frankie took the quote from Indiana Jones: thatbelongsinamuseum.com – people can nominate things that should be in a museum.
5. pricelessartefact.com – [crowdsourcing object evaluation?] – comparing objects to see which is the most valuable, however 'valuable' is defined.
[Except that possibly opens the museum to further risk of having stuff nicked to order]

Fiona Romeo, 'Different ways of seeing online collections'
I didn't take many detailed notes for this paper, but you can see my notes on a previous presentation at Notes from 'Maritime Memorials, visualised' at MCG's Spring Conference.

Mapping – objects don't make a lot of sense about themselves, but are compelling as part of information about an expedition, or failed expedition.

They'll have new map and timeline content launching next month.

Stamen can share information about how they did their geocoding and stuff.

Giving your data out for creative re-use can be as easy as giving out a CSV file.
You always want to have an API or feed when doing any website.
The National Maritime Museum make any data set they can find without licensing restrictions and put it online for creative re-use.

[Slide on approaches to data enhancement.]
Curation is the best approach but it's time-consuming.

Fiona spoke about her experiments at the mashed museum day – she cut and paste transcript data into IBM's Many Eyes. It shows that really good tools are available, even if you don't have resources to work with a company like Stamen.

Mike Ellis presented a summary of the 'mashed museum' day held the day before.

Questions, wrap up session
Jon – always assume there (should be) an API

[A question I didn't ask but posted on twitter: who do we need to get in the room to make sure all these ideas for new approaches to data, to aggregation and federation, new types of experiences of cultural heritage data, etc, actually go somewhere?]

Paul on fears about putting content online: 'since the state of Florida put pictures of their beaches on their website, no-one goes to the beach anymore'.

Metrics:
Mike: need to go shout at DCMS about the metrics, need to use more meaningful metrics especially as thinking of something like APIs
Jon: watermark metadata… micro-marketing data.
Fiona: send it out with a wrapper. Make it embeddable.

Question from someone from Guernsey Museum about images online: once you've downloaded your nice image its without metadata. George: Flickr like as much data in EXIF as possible. EXIF data isn't permanent but is useful.

Angela Murphy: wrappers are important for curators, as they're more willing to let things go if people can get back to the original source.

Me, referring back to the first session of the day: what were Lee Iverson's issues with the keynote speech? Lee: partly about the role of institution like the BBC in modern space. National broadcaster should set social common ground, be a fundamental part of democratic discussion. It's even more important now because of variety of sources out there, people shutting off or being selective about information sources to cope with information overload. Disparate source mean no middle ground or possibility of discussion. BBC should 'let it go' – send the data out. The metric becomes how widely does it spread, where does it show up? If restricted to non-commercial use then [strangling use/innovation].

The 'net recomender' thing is a flawed metric – you don't recommend something you disagree with, something that is new or difficult knowledge. What gets recommended is a video of a cute 8 year old playing Guitar Hero really well. People avoid things that challenge them.

Fiona – the advantage of the 'net recomender' is it's taking judgement of quality outside originating institution.

Paul asked who wondered why 7 – 8 on scale of 10 is neutral for British people, would have thought it's 5 – 6.

Angela: we should push data to DCMS instead of expecting them to know what they could ask for.

George: it's opportunity to change the way success is measured. Anita Roddick says 'when the community gives you wealth, it's time to give it back'. [Show, don't tell] – what would happen if you were to send a video of people engaging instead of just sending a spreadsheet?

Final round comments
Fiona: personal measure of success – creating culture of innovation, engagement, creating vibrant environment.

Paul: success is getting other people to agree with what we've been talking about [at the mashed museum day and conference] the past two days. [yes yes yes!] A measure of success was how a CEO reacted to discovering videos about their institution on YouTube – he didn't try to shut it down, but asked, 'how we can engage with that'

Ross on 'take home' ideas for the conference
Collections – we conflate many definitions in our discussions – images, records, web pages about collections.

Our tone has changed. Delivery changed – realignment of axis of powers, MLA's Digital portfolio is disappearing, there's a vacuum. Who will fill it? The Collections Trust, National Museum Directors' Conference? Technology's not a problem, it's the cultural, human factors. We need to talk about where the tensions are, we've been papering over the cracks. Institutional relationships.

The language has changed – it was about digitisation, accessibility, funding. Three words today – beauty, poetry, life. We're entering an exciting moment.

What's the role of the Museums Computer Group – how and what can the MCG do?

'Sector-wide initiatives' at 'UK Museums on the Web Conference 2008'

Session 2, 'Sector-wide initiatives', of the UK Museums on the Web Conference 2008 was chaired by Bridget McKenzie.

In the interests of getting my notes up quickly I'm putting them up pretty much 'as is', so they're still rough around the edges. There are quite a few sections below which need to be updated when the presentations or photos of slides go online. Updated posts should show in your RSS feed but you might need to check your settings.

[I hope Bridget puts some notes from her paper on her blog because I didn't get all of it down.]

The session was introduced as case studies on how cross institutional projects can be organised and delivered. She mentioned resistance to bottom-up or experimental approach, institutional constraints; and building on emerging frames of web.

Does the frame of 'the museum' make sense anymore, particularly on the web? What's our responsibilities when we collaborate? Contextual spaces – chance to share expertise in meaningful ways.

It's easy to revert to ways previous projects have been delivered. Funding plans don't allow for iterative, new and emergent technologies.

Carolyn Royston and Richard Morgan, V&A and NMOLP.
The project is funded by the 'invest to save' program, Treasury.

Aims:
Increase use of the digital collections of the 9 museums (no new website)
No new digitisation or curatorial content.
Encourage creative and critical use of online resources.
[missed one]
Sustainable high-quality online resource for partners.

The reality – it's like herding cats.

They had to address issue of partnership to avoid problems later in project.

Focussed on developing common vision, set of principles on working together, identify things uniquely achievable through partnership, barriers to success, what added value for users.

Three levels of barriers to success – one of working in an inter-museum collaborative way, which was first for those nationals; organisational issues – working inter-departmentally (people are learning or web or whatever people and not used to working together); personal issues – people involved who may not think they are web or learning people.

These things aren't necessary built in to project plan.

Deliverables: web quests, 'creative journeys', federated search, [something I missed], new ways of engaging with audiences.

Web Quests – online learning challenge, flexible learning tool mapped to curriculum. They developed a framework. It supports user research, analysis and synthesis of information. Users learn to use collections in research.

Challenges: creating meaningful collection links; sending people to collections sites knowing that content they'd find there wasn't written for those audiences; provide support for pupils when searching collections. Sustainable content authoring tool and process.

[I wondered if the Web Quest development tools are extendible, and had a chance to ask Carolyn in one of the breaks – she was able to confirm that they were.]

Framework stays on top to support and structure.

Creative journeys:
[see slide]

They're using Drupal. [Cool!]

[I also wondered about the user testing for creative journeys, whether there was evidence that people will do it there and not on their blogs, Zotero, in Word documents or hard drives – Carolyn also had some information on this.]

Museums can push relevant content.

What are the challenges?
How to build and sustain the Creative Journeys (user-generated content) communities, individually and as a partnership?
Challenge to curatorial authority and reputation
Work with messiness and complexity around new ways of communicating and using collections
Copyright and moderation issues

But partners are still having a go – shared risk, shared success.

Federated search
Wasn't part of original implementation plan
[slide on reasons for developing]
Project uses a cross collection search, not a cross collection search project. The distinction can be important.

The technical solution was driven by project objectives [choices were made in that context, not in a constraint-free environment.]

Richard, Technical Solution
The back-end is de-coupled from front end applications
A feed syndicates user actions.

Federated search – a system for creating machine readable search results and syndicating them out.
Real time search or harvester. [IMO, 'real time' should always be in scare quotes for federated searches – sometimes Google creates expectations of instantaneous results that other searches can't deliver, though the difference may only be a matter of seconds.]

Data manipulation isn't the difficult bit

Creative Journeys – more machine readable data

Syndicated user interactions with collections.
Drupal [slide]

Human factor – how to sell to board
Deploy lightweight solutions. RAD. Develop in house, don't need to go to agency.

[I'd love it if the NMOLP should have a blog, or a holding page, or something, where they could share the lessons they've learnt, the research they've done and generally engage with the digital museum community. Generally a lot of these big infrastructure projects would benefit from greater transparency, as scary as this is for traditional organisations like museums. The open source model shows that many eyeballs mean robust applications.]

Jeremy Ottevanger and Europeana/the European Digital Library
[I have to confess I was getting very hungry by this point so you might get more detailed information from Jeremy's blog when he adds his notes.]
Some background on his involvement in it, hopes and concerns.
"cross-domain access to Europe's cultural heritage"
Our content is more valuable together than scattered around.

Partnership, planning and prototyping
Not enough members from the UK, not very many museums.
Launch November this year
Won't build all of planned functionality – user-generated content and stuff planned but not for prototype.

Won't build an API or all levels of multiple linguality (in first release). Interface layer may have 3 or 4 major languages; object metadata (maybe a bit) and original content of digitised documents.

Originals on content contributors site, so traffic ends up there. That's not necessarily clear in the maquette (prototype). [But that knowledge might help address some concerns generally out there about off-site searches]

Search, various modes of browsing, timeline and stuff.

Jeremy wants to hear ideas, concerns, ambitions, etc to take to plenary meeting.

He'd always wanted personal place to play with stuff.

[Similarly to my question above, I've always wondered whether users would rely on a cultural heritage sector site to collate their data? What unique benefits might a user see in this functionality – authority by association? live updates of data? Would they think about data ownership issues or the longevity of their data and the reliability of the service?]

Why are there so few UK museums involved in this? [Based on comments I've heard, it's about no clear benefits, yet another project, no API, no clear user need] Jeremy had some ideas but getting in contact and telling him is the best way to sort it out.

Some benefits include common data standards, a big pool of content that search engines would pay attention to in a way they wouldn't on our individual sites. Sophisticated search. Will be open source. Multi-lingual technology.

Good news:
"API was always in plans".

EDLocal – PNDS. EU projects will be feeding in technologies.

Bad news: API won't be in website prototype. Is EDLocal enough? Sustainability problems.
'Wouldn't need website at all if had API'. Natural history collections are poorly represented.

Is OAI a barrier too far? You should be able to upload from spreadsheet. [You can! But I guess not many people know this – I'm going to talk to the people who coded the PNDS about writing up their 'upload' tool, which is a bit like Flickr's Uploadr but for collections data.]

Questions
Jim O'Donnell: regarding the issue of lack of participation. People often won't implement their own OAI repository so that requirement puts people off.

Dan Zambonini: aggregation fatigue. 'how many more of these things do we have to participate in'. His suggestion: tell museums to build APIs so that projects can use their data, should be other way around. Jeremy responded that that's difficult for smaller museums. [Really good point, and the PNDS/EDL probably has the most benefits for smaller museums; bigger museums have the infrastructure not to need the functionality of the PNDS though they might benefit from cross-sector searching and better data indexing.]

Gordon McKenna commented: EDLocal starts on Wednesday next week, for three years.

George Oates: what's been most surprising in collaboration process? Carolyn: that we've managed to work together. Knowledge sharing.

Are small museums the long tail?

On the way home from the Semantic Web Think Tank last week (see previous post), I suddenly thought: are small or specialised museums the long tail?

Each museum by itself would represent a tiny proportion of the overall use of museum collections online, but if you put all that usage together, would their collections in fact have a higher rate of use than those of more 'popular' museums?

At the moment I don't think there's any way to find out, because so many small or specialised museums don't have collections online, through a lack of expertise, digitisation resources or an easy-to-use publication infrastructure. Still, it's an interesting question.