Category: Technology

Oh noes, a FAIL! Notes from the unconference session on 'failure' at MW2009

These are my really rough notes from the unconference session at Museums and the Web, written up quickly in order to capture the essence of the discussion and open it up for comment.

Susan Chun, Dana Mitroff Silvers, Bruce Wyman and I began and were later joined by Seb Chan and Jennifer Trant.

I explained my motivation in suggesting the session – intelligent, constructive failure is important. Finding ways to create a space for that conversation isn't something we do well at the moment.

Susan started the conversation by pointing out that there were different definitions or types of failure. Defining 'failure' more precisely is useful.

Types of failures include: over budget, badly implemented, badly specified, future failures.

Dana pointed out that we needed to define success as well as defining failure. A more nuanced understanding of failure is important, especially when hoping to encourage more people to talk about failure. Discussion about choosing the right metrics for success – the right metrics may vary depending on whether you're a funder or a department or whoever.

Funding models can set you up for failure.

Bruce pointed out that it's not the failure that matters, it's what you do with the failure.

Some apparent failures may not really be failures.

Are you funding the process or the product?

Not having the mechanism for exposing the knowledge is a failure.

The definitions of failure and success need to include the net gain for an organisation or in new/improved processes as well as the product.

What kind of environment is needed so that people can publish judgements of their own success or failure?

Susan suggested the MCN project registry would be a good place for this information.

What if it was routine to talk about what failed or succeeded in each project? Funding should reward people who talk about failures. Discussion about space for reflection on 'lessons learned' in project summation.

Agency is important – you talk about the failures of your own projects, other people don't dob you in.

Dana – talking about failures in a project should be a normal part of MW papers.

Label it 'lessons learned', not 'failure'.

Susan – [Remove roadblocks about what happens if funders hear you think your project failed in some way -] Talk to funders about requiring an examination or reflection of each project for failure in the same way the issue of open source development was tackled. Pro-active approach!

Me: when you're putting in for funding, you should have to show that you've talked to people with similar projects about the lessons they learned.

Susan – put ILMS (Institute of Museum and Library Services) reports online. [A small but practical thing to do]. Change the culture of secrecy.

Funding can be a carrot and a stick. Without that, institutional change is hard.

Points of resistance (some summing up):
understanding how to define failure/success
culture of secrecy
fear of exposure to funders
lacking the jargon to describe failure (which would also help normalise the process of discussing it openly)

Jennifer – if there aren't any negative consequences, why can't you talk about it?

General discussion about the need for early, continual dialogue about projects. It's difficult to talk about failures if you're not already talking about the project. Paraphrasing Seb -talking about it already in an informal context, like a blog, may help here.

Iterative, transparent reporting is important. It also helps other people talk about failures.

Susan – other causes of failures are project that never happened. Whether they missed their time, didn't get funding, whatever. Consider those as failures too, and talk about them. Everyone benefits, whether that's the person with the great idea that never got to see it happen, or people who've built on it later.

Talk about nascent projects. Exposing them to comment early can help prevent failure. Like the old crack about voting, public discussion about projects should happen early and often!

Hoarding ideas is pointless.

We need a template for talking about failure. Prompts or questions for consideration.

It's not just overall project failures, it can include institutional, departmental or structural failures.

Dana suggested confessional sessions, perhaps at the next Museums and the Web conference. Jennifer and Seb took it up, suggesting YouTube captures with disguised voices and silhouettes to make it easier, and encouraging discussion of failures by type or theme.

Discussion about the role of commentators, respondents in sessions. The voice of the one that didn't work.

Find an acceptable form of critical questions so that people can help prevent other projects failing, make the most of the experience out there.

Putting my money where my mouth is, one final comment from Seb was about a possible failure of the unconference sessions in not getting people together again at the end to report back. This was received constructively, and might happen during the final plenary.

Yay! Three Lazy Geeks shortlisted for dev8D prize

We're in the top five, whoop!

The reviewers said, "Really comprehensive treatment of the problem and associated issues. Worth pursuing I think… As a solution this is a good idea and was produced by genuine collaboration at the Dev8D event."

So a short but happy developer post from me. The whole experience was lots of fun, and it would never have worked without Ian Ibbotson and Pete Sefton. I think the thing that I like most about it is that it not only re-uses existing tools, it fits with how people already work. It's not "this application will change your life, but first you have to change your life". I know that the (mostly junior) academics I've mentioned it to have loved the idea, so it might have real users if it was developed, which would be lovely.

Tim Berners-Lee at TED on 'database hugging' and linked data

This TED talk by Tim Berners-Lee: The next Web of open, linked data is worth watching if you've been 'wondering whatever happened to the semantic web?', or what this 'linked data' is about all.

I've put some notes below – I was transcribing it for myself and thought I might as well share it. It's only a selection of the talk and I haven't tidied it because they're not my words to edit.

Why is linked data important?

Making the world run better by making this data available. If you know about some data in some government department you often find that, these people, they're very tempted to keep it, to hug your database, you don't want to let it go until you've made a beautiful website for it. … Who am I to say "don't make a website…" make a beautiful website, but first, give us the unadulterated data. Give us the raw data now.

You have no idea, the number of excuses people come up with to hang onto their data and not give it to you, even though you've paid for it.

Communicating science over the web… the people who are going to solve those are scientists, they have half-formed ideas in their head, but a lot of the state of knowledge of the human race at the moment is in database, currently not sharing. Alzheimer's scientists … the power of being able ask questions which bridge across different disciplines is really a complete sea-change, it's very, very important. Scientists are totally stymied at the moment, the power of the data that other scientists have collected is locked up, and we need to get it unlocked so we can tackle those huge problems. if I go on like this you'll think [all data from] huge institutions but it's not. [Social networking is data.]

Linked data is about people doing their bit to produce their bit, and it all connecting. That's how linked data works. … You do your bit, everybody else does theirs. You may not have much data yourself, to put on there, but you know to demand it.

It's not just about the number of places where data comes. It's about connecting it together. When you connect it together you get this power… out of it. It'll only really pay off when everybody else has done it. It's called Linked Data, I want you to make it, I want you to demand it.

Crowd-sourcing the translation of museum content into sign language?

We've been thinking about crowd-sourcing some British Sign Language (BSL) content for the Science Museum for a while now, particularly as we're running events with BSL interpreters and a new site ('Brought to Life') with some BSL content is due to launch in March. This post is both an attempt to think through some of the issues, and a question open to all – what do you think?

The idea
There are two related options – asking the public to share their translations of English text on the Science Museum websites or galleries into BSL with us, or asking people to contribute new content in BSL. Translations could include content like object captions (to view online or download to portable devices to take into the museum), exhibition information and interpretation, instructions for games like Launchpad – any existing content online or in the galleries.

Why it could be useful
Linda Ellis gave a presentation at the UK Museums Computer Group (MCG) meeting on 'Unheard Stories – Improving access for Deaf visitors' where she pointed out the distinctions between 'deaf' and 'Deaf', including that Deaf people use sign language as their first language and might not know English while deaf people probably become deaf later in life and English is their first language. Linda also said that Deaf people are one of the most excluded groups in our society. Deaf visitors surveyed for the Wolverhampton Arts and Museums Service said they wanted: concise written information; information in BSL; to explore exhibits independently; stories about local people and museum objects; events just for Deaf people (and dressing up, apparently).

(More notes on Linda's presentation and a link to her slides are in this earlier post).

I saw a great example of BSL content in museums at the 2009 Jodi Awards. The British Museum worked with the Frank Barnes School and media company Remark on a project where young deaf people produced signed curriculum resources for young deaf people. You can find out more and watch the videos at British Sign Language videos about the Museum.

Video goes mainstream?
One uncertainty is whether possible contributors would be comfortable creating and uploading video. The popularity of products like 'You Tube ready' digital compact cameras and the Flip would suggest that consumers are comfortable with the idea of creating and sharing video online.

The 2008 Horizon Report suggested 'grassroots video' will be adopted in one year or less:

Video is everywhere—and almost any device that can access the Internet can play (and probably capture) it. From user-created clips and machinima to creative mashups to excerpts from news or television shows, video has become a popular medium for personal communication. Editing and distribution can be done easily with affordable tools, lowering the barriers for production. Ubiquitous video capture capabilities have literally put the ability to record events in the hands of almost everyone. Once the exclusive province of highly trained professionals, video content production has gone grassroots.

In terms of understanding the context and perhaps expecting video online, a report The Valley looks towards 2009 in the BBC quotes Jim Patterson, product manager at YouTube, saying:

"This generation of users utilize the web differently and consume video differently. They grew up in an environment where digital, interactive media was ubiquitous. It has shaped how they use the web."

And Mr Patterson said this new video generation has also shaped the very nature of how YouTube is being used.

"Comscore is estimating that YouTube is the second largest search engine," he said.

"To this cohort, YouTube is their search engine. YouTube 'is' the web. Seeking the answer to any question, they prefer that the result be expressed as a video, so they go to YouTube."

That last point – "YouTube is their search engine. YouTube 'is' the web" – is pretty damn important, regardless of any other issues around museum content.

My questions

  • Am I imagining a need that isn't there? Are there enough people with British Sign Language as a first language who are interested in content at the Science Museum to make the project worthwhile? Is BSL content about particular objects or exhibitions something d/Deaf visitors would find useful?
  • Would anyone out there be interested in creating this content?
  • Is there enough acceptance of internet video? Is it easy enough for the public to produce and upload their own videos?

What do you think?

Catalhoyuk question

This will only be relevant to the archaeologists, I guess, but it has occurred to me to ask – what would you like to see in the Catalhoyuk archive reports? What information would either be useful or satisfy your curiosity?

In a wider sense, what can we (as IT geeks in the cultural heritage sector) learn from each other? What are we too scared to ask in case it's a stupid question, or because it seems too obscure? What don't we share because we assume that everyone else knows it already?

Final diary entry from Catalhoyuk

I'm back in London now but here goes anyway:

August 1
My final entry of the season as I'm on the overnight train from Cumra to Istanbul tonight. After various conversations on the veranda I've been thinking about the intellectual accessibility of our Catalhoyuk data and how that relates to web publication and this entry is just a good way to stop these thoughts running round my head like a rogue tune.

[This has turned into a long entry, and I don't say anything trivial about the weather or other random things so you'd have to be pretty bored to read it all. Shouldn't you be working instead?]

Getting database records up on the website isn't hard – it's just a matter of resources. The tricky part is providing an interesting and engaging experience for the general visitor, or a reliable, useable and useful data set for the specialist visitor.

At the moment it feels like a lot of good content is hidden within the database section of the website. When you get down to browsing lists of features, there's often enough information in the first few lines to catch your interest. But when you get to lists of units, even the pages with some of the unit description presented alongside the list, you start to encounter the '800 lamps' problem.

[A digression/explanation – I'm working on a website at the Museum of London with a searchable/browsable catalogue of objects from Roman Londinium. One section has 800 Roman oil lamps – how on earth can you present that to the user so they can usefully distinguish between one lamp and another?]

Of course, it does depend on the kind of user and what they want to achieve on the Londinium site – for some, it's enough to read one nicely written piece on the use of lamps and maybe a bit about what different types meant, all illustrated with a few choice objects; specialist users may want to search for lamps with very particular characteristics. Here, our '800 lamps' are 11846 (and counting) units of archaeology. The average user isn't going to read every unit sheet, but how can they even choose one to start with? And how many will know how to interpret and create meaning from what they read about the varying shades of brown stuff? Being able to download unit sheets that match a particular pattern – part of a certain building, ones that contain certain types of finds, units related to different kinds of features – is probably of real benefit to specialist visitors, but are we really giving those specialist visitors (professional or amateur) and our general visitors what they need? I'm not sure a raw huge list of units or flotation numbers is of much help to anyone – how do people distinguish between one thumbnail of a lamp or one unit number and another in a useful and meaningful way? I hope this doesn't sound like a criticism of the website – it's just the nature of the material being presented.

The variability of the data is another problem – it's not just about data cleaning (though the 'view features by type' page shows why data cleaning is useful) – but about the difference between the beautiful page for Building 49 and rather less interesting page for Building 33 (to pick one at random). If a user lands on one of the pages with minimal information they may never realise that some pages have detailed records with fantastic plans and photos.

So there are the barriers to entry that we might accidentally perpetuate by 'hiding' the content behind lists of numbers; and there is the general intellectual accessibility of the information to the general user. Given limited resources, where should our energies be concentrated? Who are our websites for?

It's also about matching the data and website functionality to the user and their goals – the excavation database might not be of interest to the general user in its most raw form, and that's ok because it will be of great interest to others. At a guess, the general public might be more interested in finds, and if that's the case we should find ways to present information about the objects with appropriate interpretation and contextualisation, not only to present information about the objects but also to help people have a more meaningful experience on the site.

I wonder if 'team favourite' finds or buildings/spaces/features could be a good way into the data, a solution that doesn't mean making some kinds of finds or some buildings into 'treasure' and more important than others. Or perhaps specialists could talk about a unit or feature they find interesting – along the way they could explain how their specialism contributes to the archaeological record (written as if to an intelligent thirteen year old). For example, Flip could talk about phytoliths, or Stringy could talk about obsidian, and what their finds can tell us.

Proper user evaluation would be fabulous, but in the absence of resources, I really should look at the stats and see how the site is used. I wonder if I could do a surveymonkey thing to get general information from different types of users? I wonder what levels of knowledge our visitors have about the Neolithic, about Anatolian history, etc. What brings them to the website? And what makes them stick around?

Intellectual accessibility doesn't only apply to the general public – it also applies to the accessibility of other team's or labs content. There are so many tables hidden behind the excavation and specialist database interfaces – some are archived, some had a very particular usage, some are still actively used but still have the names of long-gone databases. It's all very well encouraging people to use the database to query across specialisms, but how will they know where to look for the data they need? [And if we make documentation, will anyone read it?]

It was quite cool this morning but now it's hot again. Ha, I lied about not saying anything trivial about the weather! Now go do some work.
(Started July 29, but finally posted August 1)

Catalhoyuk diaries: In the absence of real world content…

…more diary entries from Catalhoyuk.
July 24, started late at night in tent. [but posted as July 25, 2007]
Spent some time whizzing things around in ArcGIS the past few days. I never get to play with GIS at work so it's quite fun. I need to talk to Cord and Dave about what views/tables they need in the database to link in the excavation and finds data. It's a shame we didn't get to experiment with bringing data across before Cord left but hopefully Dave and I can do a 'proof of concept' that she can play with when she gets back to site. Maybe skellies or X-finds, or just basic unit information as a first go.

Today was going to be a solid day of programming, but the power was out for three hours last night and two hours of lab time this morning, so I'm still catching up on the stuff I was going to finish last night.

Last night I wrote myself a note from the geek perspective about "I think the challenge of Catal is combining the reflexive, the uncertain, the indeterminate, with the rigorous requirements of structured recording in a database; and perhaps more importantly, convincing people that it's possible to design to allow for uncertainty and for multivocality" but in the light of day that sounds like pretentious tosh that could only have been thought up in the middle of the night. Well done me.

July 30, middle of the day.
It's really been quite hot, though just now a change is coming through and it's getting cooler. I'm such a (lab, not southern) jessie, I can't imagine what it's like working up on the mound – apparently it's been 48C in the south shelter. The feel of the coming storm reminds me of Melbourne but I bet the heat won't break after the storm like it would at home. I hope my borrowed tent doesn't blow away.

Very frustrated by the power cuts. I feel like I'm losing lots of work time to them. My list of things to do is getting longer and my time is getting shorter. I wish it was the other way around.

I've been documenting some of the database tables in preparation for an informal tutorial on database querying tomorrow. I say 'informal' but that's really just cos I don't have time to prepare so I'll wing it. Hopefully people will have some good examples. I think it'll also point out where we need to make improvements – putting all the relationships into the central database is probably the first thing to do, so that they automatically come through into the AllTables database. This will make it a lot easier for people to join tables as some joins will be created auto-magically for them. It's probably not worth documenting all the tables at the field level, but there are some (where the data type is different between old databases, for example) where it would be useful. The descriptions could also serve as synonyms to help people find the tables they need for their queries.

Update from Catalhoyuk

In the absence of real updates, I thought I'd post some of my site diary entries. The power has been very dodgy the past few days so I might do a big catch up on email and whatever in Konya on Friday (our day off). Interestingly, Blogger has decided to present me the site in Turkish, presumably based on IP location, because the language settings on the browser are English-only. So if things go a bit strange it's because I can't really see what I'm doing.

19/07/2007
My first day on site this season. I feel like I've been pole-axed with tiredness after the trip out here, so I'm concentrating on catching up on Sarah's documentation [for her work on site this year] and generally remembering how everything works.

24/07/2007
Just had a random thought, though it's a shame I didn't think of it at the start so we could tell everyone who's been on site over the season – any blog posts, photos or videos, blah blah blah, could use the same tag (like 'catal07') on public content, so it's easier to find everything from this season regardless of where it's held.

[And now that I'm posting this on a blog I suppose I should do that myself]

Tuesday, July 24, 2007

I keep bouncing between looking at the Figurines and Ceramics databases.
I've been nabbing poor Chris whenever he comes anywhere near the computer room and asking questions about heat treatment and cores; he's been very patient.

We had a long meeting in the cafe on Sunday, and I re-jigged the recording structures afterwards. I think possibly last year's structure was too ambitious, given the time constraints on everyone – not only for building it, but for mapping data from old structures to the new and mostly for the time it takes to simply record the objects as it went into lots of technical detail that probably isn't sustainable at the moment.

With that in mind, I tipped the recording model on its head so that it's much more about observation than interpretation at this stage, particularly for colours and the various things that variations in colour indicate. For example, rather than breaking heat exposure down into manufacture, use, other events or post-deposition, for the moment it's enough to record that it's present. I've designed the forms to allow people to record the probable type of heat exposure (and how certain they are about it) if there's evidence to support it, but if there's no evidence either way they don't have to record a probable reason. The structures can be extended as we find out more about the raw materials around the site – I think they might change views on the intentionality represented by the presence of various inclusions.

I've spent the day reviewing the ceramics database structures with a view to normalising them, and also to fitting them into the shared clay recording system. It's a continuation of work from previous seasons but with the added pressure(?) challenge(?) that other teams will be using versions of the ceramics databases soon too, so it's really important to get the data structures right. Nurcan has been really helpful and her explanations of some of the changes have helped me think about the best solutions to her recording issues.

Journalists were out yesterday, we had our photo taken under the Catalhoyuk sign near the gate. Apparently it'll be in Wednesday's papers. I wonder if people in London could pick up copies in the off-licences around Green Lanes. 'Famous in Turkey' – sounds like a band name.

It's funny how the diary entries are starting to read like blog entries, and in a way they seem to be functioning a bit like a blog too, with people commenting on each other's entries. I almost feel like I should add a field so people can record which diary entry they're writing about alongside the units, etc, but would that be far too self-referential?

When the database goes back to London and is put on the web I think we should put an 'AddThis' button on the various diary, finds and excavation pages so people can add pages to social bookmarking sites, blogs, etc. If we sign up for an account we can see how it's used – I wonder how much activity that kind of 'passive' use would see compared to 'active' use like commenting on finds or excavation data. I really need to find out more about the barriers to participation for actively creating content. I'll suggest the 'AddThis' thing to Ian and Shahina if I get a chance.

I'm starting to really wish I'd had my hair cut before I left London because it's taken on a life of its own. Not that it really matters out here, I guess.

I'm off to Catalhoyuk for two weeks

I'm off to Çatalhöyük, Turkey, for two weeks, for the usual database analysis/design/development. I don't imagine I'll have any time to post updates, so you can make do with photos from previous years in the meantime.

www.flickr.com

Catalhoyuk 2004, 2005, 2006 photoset

Some of the history of the Catalhoyuk database

I was going to post this on the Catalhoyuk blog but authentication isn't working right now. So, I'll post it here and move it over when it's working again.

Just in case you thought nothing happened during the off-season…

A lot of this information is contained in the Archive Reports but as the audience for those is probably more specialised than the average reader of this blog, I thought it might be interesting to talk about them here.

When MoLAS first became involved with the project, there were lots of isolated Microsoft Access 2000 databases for excavation, finds and specialist data. I could see that the original database design and structure was well structured and much valuable work had been done on the database previously. However, some problems had arisen over the years as the database grew and different specialists brought their own systems based on a mixture of applications and platforms.

It was difficult for specialist databases to use live field or excavation data because it wasn't available in a single central source. It had also become almost impossible to run queries across excavation seasons or areas, or produce multi-disciplinary analysis , as there were disparate unrelated databases for each area of study. Within many specialisms the data set has been broken up into many different files – for example, the excavation database was split into teams and some teams were creating separate files for different years.

In many cases, referential integrity was not properly enforced in the interface or database structure. While the original database structures included tables to supply lists of values to enable controlled vocabularies, the interfaces were using static rather than dynamic menus on data entry interfaces. Primary and/or foreign keys were not implemented in some databases, leading to the possibility of multiple entries, anomalous data or incorrect codes being recorded. There was little or no validation on data entry.

IBM generously donated two new servers, one for use on site and the other for the Cambridge office. This meant that we were able to install Microsoft SQL Server 2000 to use as a single backend database and start re-centralising the databases. This meant re-combining the disparate datasets into a single, central database, and reconfiguring the Access forms to use this new centralised backend.

Centralising and cleaning the data and interfaces was a bit of a slog (covered in more detail in the archive reports), and even now there are still bits and pieces to be done. I guess this shows the importance of proper database design and documentation, even when you think a project is only going to be small. I'm sure there was documentation originally, so I guess this also shows the importance of a good archiving system!

Unfortunately, because the 'business logic' of the database applications wasn't documented (if there was documentation it'd been lost over time) we couldn't re-do the existing forms in another application (like web forms) without losing all the validation and data entry rules that had been built up over time in response to the specialists' requirements. As usual in the world of archaeology, limited resources meant this wasn't possible at that stage. A lot of the application logic seemed to be held in the interfaces rather than in the relationships between tables, which meant a lot of data cleaning had to be done when centralising the databases and enforcing relationships.

As the 2004 Archive Report says, "The existing infrastructure was Microsoft Access based, and after consideration for minimal interruption to existing interfaces, and for the cost to the project of completely redeveloping the forms on another platform, these applications were retained."

Luckily, we're not tied to Access for new application development, and new and future database applications are created as HTML, eliminating any platform/OS compatibility issues.

This means that we can get on with more exciting things in the future! I'll post about some of those ideas soon.

In the meantime, check out the public version of the web interface to the Çatalhöyük database.

[Originally published on http://www.catalhoyuk.com/blog/, January 24, 2007]