Tag: research

These are a few of my favourite (audience research) things

On Friday I popped into London to give a talk at the Art of Digital meetup at the Photographer's Gallery. It's a great series of events organised by Caroline Heron and Jo Healy, so go along sometime if you can. I talked about different ways of doing audience research. (And when I wrote the line 'getting to know you' it gave me an earworm and a 'lessons from musicals' theme). It was a talk of two halves. In the first, I outlined different ways of thinking about audience research, then went into a little more detail about a few of my favourite (audience research) things.

There are lots of different ways to understand the contexts and needs different audiences bring to your offerings. You probably also want to test to see if what you're making works for them and to get a sense of what they're currently doing with your websites, apps or venues. It can help to think of research methods along scales of time, distance, numbers, 'density' and intimacy. (Or you could think of it as a journey from 'somewhere out there' to 'dancing cheek to cheek'…)

'Time' refers to both how much time a method asks from the audience and how much time it takes to analyse the results. There's no getting around the fact that nearly all methods require time to plan, prepare and pilot, sorry! You can run 5 second tests that ask remote visitors a single question, or spend months embedded in a workplace shadowing people (and more time afterwards analysing the results). On the distance scale, you can work with remote testers located anywhere across the world, ask people visiting your museum to look at a few prototype screens, or physically locate yourself in someone's office for an interview or observation.

Numbers and 'density' (or the richness of communication and the resulting data) tend to be inversely linked. Analytics or log files let you gather data from millions of website or app users, one-question surveys can garner thousands of responses, you can interview dozens of people or test prototypes with 5-8 users each time. However, the conversations you'll have in a semi-structured interview are much richer than the responses you'll get to a multiple-choice questionnaire. This is partly because it's a two-way dialogue, and partly because in-person interviews convey more information, including tone of voice, physical gestures, impressions of a location and possibly even physical artefacts or demonstrations. Generally, methods that can reach millions of remote people produce lots of point data, while more intimate methods that involve spending lots of time with just a few people produce small datasets of really rich data.

So here are few of my favourite things: analytics, one-question surveys, 5 second tests, lightweight usability tests, semi-structured interviews, and on-site observations. Ultimately, the methods you use are a balance of time and distance, the richness of the data required, and whether you want to understand the requirements for, or measure the performance of a site or tool.

Analytics are great for understanding how people found you, what they're doing on your site, and how this changes over time. Analytics can help you work out which bits of a website need tweaking, and for measuring to see the impact of changes. But that only gets you so far – how do you know which trends are meaningful and which are just noise? To understand why people are doing what they do, you need other forms of research to flesh them out. 
One question surveys are a great way of finding out why people are on your site, and whether they've succeeded in achieving their goals for being there. We linked survey answers to analytics for the last Let's Get Real project so we could see how people who were there for different reasons behaved on the site, but you don't need to go that far – any information about why people are on your site is better than none! 
5 second tests and lightweight usability tests are both ways to find out how well a design works for its intended audiences. 5 second tests show people an interface for 5 seconds, then ask them what they remember about it, or where they'd click to do a particular task. They're a good way to make sure your text and design are clear. Usability tests take from a few minutes to an hour, and are usually done in person. One of my favourite lightweight tests involves grabbing a sketch, an iPad or laptop and asking people in a café or other space if they'd help by testing a site for a few minutes. You can gather lots of feedback really quickly, and report back with a prioritised list of fixes by the end of the day. 
Semi-structured interviews use the same set of questions each time to ensure some consistency between interviews, but they're flexible enough to let you delve into detail and follow any interesting diversions that arise during the conversation. Interviews and observations can be even more informative if they're done in the space where the activities you're interested in take place. 'Contextual inquiry' goes a step further by including observations of the tasks you're interested in being performed. If you can 'apprentice' yourself to someone, it's a great way to have them explain to you why things are done the way they are. However, it's obviously a lot more difficult to find someone willing and able to let you observe them in this way, it's not appropriate for every task or research question, and the data that results can be so rich and dense with information that it takes a long time to review and analyse. 
And one final titbit of wisdom from a musical – always look on the bright side of life! Any knowledge is better than none, so if you manage to get any audience research or usability testing done then you're already better off than you were before.

[Update: a comment on twitter reminded me of another favourite research thing: if you don't yet have a site/app/campaign/whatever, test a competitor's!]

If an app is the answer…

…what was the question?*  And seriously, what questions should museums ask before investing in a mobile or tablet app?

First, some background. In We’re not ‘appy. Not ‘appy at all., Tom Loosemore of the Government Digital Service (GDS) gives examples of the increases in mobile traffic to UK government sites, including up to 60% mobile visits to site with complex transactions like booking driving tests.

For a cultural heritage perspective: as part of the Culture24 Let's Get Real project, I looked at the percentage of mobile visits (and visitors) to 22 cultural websites for Jan 1, 2012 – September 2, 2012 (an extended time to try and reduce the effect of the Olympics) and found that on average, museums and arts organisations were already seeing an average of 20% mobile visits.  I also reported the percentage change from the same period last year so that people could get a sense of the velocity of change: on average there was a 170% increase in mobile visits to cultural websites compared to the same period in 2011.

We'll re-run the stats when writing the final report in July, but in the meantime, there's a recent significant increase in web traffic from tablet devices to take into account.  In BBC iPlayer: tablet viewing requests nearly double in two months the Guardian reported BBC figures:

"Tablets' share of total iPlayer requests grew from just 6% (TV only: 7%) in January 2012 to 10% (12%) in November and 15% (18%) last month. Smartphone requests have seen similar growth from 6% (TV only: 6%) of the total a year ago to 16% (18%) in January. [… ]A spokesman said it is thought the rise of the "phablet" – smartphones that are almost as big as a tablet, such as the Samsung Note – that have driven the surge."

Some museums are reporting seeing 40-50% increases in tablet traffic in the past few months. So, given all that, are apps the answer?  Over to Loosemore:

"Our position is that native apps are rarely justified. […] Apps may be transforming gaming and social media, but for utility public services, the ‘making your website adapt really effectively to a range of devices’ approach is currently the better strategy. It allows you to iterate your services much more quickly, minimises any market impact and is far cheaper to support."

Obviously there are exceptions for apps that meet particular needs or genres, but this stance is part of their Government Digital Strategy:

"Stand-alone mobile apps will only be considered once the core web service works well on mobile devices, and if specifically agreed with the Cabinet Office." 

So if your cultural organisation is considering an app, perhaps you should consider the questions the GDS poses before asking for an exemption to the requirement to just build a responsive website:

  1. Is our web service already designed to be responsive to different screen sizes? If not, why not?
  2. What is the user need that only a native/hybrid app can meet?
  3. Are there existing native/hybrid apps which already meet this user need?
  4. Is our service available to 3rd parties via an API or open data? If not, why not?
  5. Does meeting this need justify the lifetime cost of a native or hybrid app?
What questions should we add for cultural heritage, arts and educational organisations?  (My pet hate: are you creating amazing content that's only accessible to people with the right device?) And since I know I'm being deliberately provocative – what exceptions should be allowed? What apps have you seen that could only work as an app with current technology?

* I can't claim credit for the challenge 'if an app is the answer, what was the question', it's been floating around for a while now and possibly originated at a Let's Get Real workshop or conference.

'Entrepreneurship and Social Media' and 'Collaborating to Compete'

[Update: I hope the presentations from the speakers are posted, as they were all inspiring in their different ways.  Bristol City Council's civic crowdsourcing projects had impressive participation rates, and Phil Higgins identified the critical success factors as: choose the right platform, use it at the right stage, issue must be presented clearly. Joanne Orr talked about museum contexts that are encapsulating the intangible including language and practices (and recording intangible cultural heritage in a wiki) and I could sense the audience's excitement about Andrew Ellis' presentation on 'Your Paintings' and the crowdsourcing tagger developed for the Public Catalogue Foundation.]

I'm in Edinburgh for the Museums Galleries Scotland conference 'Collaborating to Compete'. I'm chairing a session on 'Entrepreneurship and Social Media'. In this context, the organisers defined entrepreneurship as 'doing things innovatively and differently', including new and effective ways of working. This session is all about working in partnerships and collaborating with the public. The organisers asked me to talk about my own research as well as introducing the session. I'm posting my notes in advance to save people having to scribble down notes, and I'll try to post back with notes from the session presentations.

Anyway, on with my notes…

Welcome to this session on entrepreneurship and social media. Our speakers are going to share their exciting work with museum collections and cultural heritage.  Their projects demonstrate the benefits of community participation, of opening up to encourage external experts to share their knowledge, and of engaging the general public with the task of improving access to cultural heritage for all.  The speakers have explored innovative ways of working, including organisational partnerships and low-cost digital platforms like social media.  Our speakers will discuss the opportunities and challenges of collaborating with audiences, the issues around authority, identity and trust in user-generated content, and they'll reflect on the challenges of negotiating partnerships with other organisations or with 'the crowd'.

You'll hear about two different approaches to crowdsourcing from Phil Higgins and Andy Ellis, and about how the 'Intangible Cultural Heritage' project helps a diverse range of people collaborate to create knowledge for all.

I'll also briefly discuss my own research into crowdsourcing through games as an example of innovative forms of participation and engagement.

If you're not familiar with the term, crowdsourcing generally means sharing tasks with the public that are traditionally performed in-house.

Until I left to start my PhD, I worked at the Science Museum in London, where I spent a lot of time thinking about how to make the history of science and technology more engaging, and the objects related to it more accessible. This inspired me when I was looking for a dissertation project for my MSc, so I researched and developed 'Museum Metadata Games' to explore how crowdsourcing games could get people to have fun while improving the content around 'difficult' museum objects.

Unfortunately (most) collections sites are not that interesting to the general public. There's a 'semantic gap' between the everyday language of the public and the language of catalogues.

Projects like steve.museum showed crowdsourcing helps, but it can be difficult to get people to participate in large numbers or over a long period of time. Museums can be intimidating, and marketing your project to audiences can be expensive. But what if you made a crowdsourcing interface that made people want to use it, and to tell their friends to use it? Something like… a game?

A lot of people play games… 20 million people in the UK play casual games. And a lot of people play museum games. Games like the Science Museum's Launchball and the Wellcome Collection's High Tea have had millions of plays.

Crowdsourcing games are great at creating engaging experiences. They support low barriers to participation, and the ability to keep people playing. As an example, within one month of launching, DigitalKoot, a game for National Library of Finland, had 25,000 visitors complete over 2 million individual tasks.

Casual game genres include puzzles, card games or trivia games. You've probably heard of Angry Birds and Solitaire, even if you don’t think of yourself as a 'gamer'.

Casual games are perfect for public participation because they're designed for instant gameplay, and can be enjoyed in a few minutes or played for hours.

Easy, feel-good tasks will help people get started. Strong game mechanics, tested throughout development with your target audience, will motivate on-going play and keep people coming back.

Here’s a screenshot of the games I made.

In the tagging game 'Dora's lost data', the player meets Dora, a junior curator who needs their help replacing some lost data. Dora asks the player to add words that would help someone find the object shown in Google.

When audiences can immediately identify an activity as a game – in this the use of characters and a minimal narrative really helped – their usual reservations about contributing content to a museum site disappear.

The brilliant thing about game design is that you can tailor tasks and rewards to your data needs, and build tutorials into gameplay to match the player’s skills and the games’ challenges.

Fun is personal – design for the skills, abilities and motivations of your audience.

People like helping out – show them how their data is used so they can feel good about playing for a few minutes over a cup of tea.

You can make a virtue of the randomness of your content – if people can have fun with 100 historical astronomy objects, they can have fun with anything.

To conclude, crowdsourcing games can be fun and useful for the public and for museums. And now we're going to hear more about working with the public… [the end!]

Interview about museum metadata games and a pretty picture

I haven't had a chance to follow up Design constraints and research questions: museum metadata games with a post about the design process for the museum metadata games I've made for my dissertation project (because, stupidly, I slipped on black ice and damaged my wrist), so in the meantime here's a link to an interview Seb Chan did with me for the Fresh+New blog, Interview with Mia Ridge on museum metadata games, and a Wordle of the tags added so far.

There have been nearly 700 turns on the games so far, which have collectively added about 30 facts (Donald’s detective puzzle) and just over 3,700 tags (Dora’s lost data).

Some of the 1,582 unique tags added so far

Design constraints and research questions: museum metadata games

Back in June I posted parts of my dissertation project outline in 'Game mechanics for social good: a case study on interaction models for crowdsourcing museum collections enhancement'. Since then, I've been getting on with researching, designing, building and evaluating museum metadata games (in my copious spare time after work, in a year when we launched three major galleries).

I'm planning to blog bits of my dissertation as I write it up so there'll be more posts over the next month, but for now I wanted to contextualise the two games I'm evaluating at the moment.  In the next post I'll talk about the changes I made after the first solid round of evaluation.

Casual games
The two games, nicknamed 'Dora' and 'Donald' are designed as casual games – something you can pick up and play for five minutes at a time.  Design goals included: an instantly playable game that provides stress relief, supports a competitive spirit (but not necessarily against other people), inherently rewarding experience, simple game play and puts 'fun before do-gooding'.  The games were designed around a specific research-based persona ('Meet Janet', pdf link) – hopefully it's exactly right for some people who are close to the persona in various ways, and quite fun for a wider group.  It won't suit everyone, not least because definitions of 'fun' and expectations around 'games' can be deeply individual.

Design constraints
The games are also designed to test ideas about the types of objects and records that can be used successfully, and the types of content people would be able to contribute about the less charismatic and emotionally accessible reaches of science, technology and social history collections – this means that some of the objects I've used are quite technical, not all the images are great and small variations on object records are repeated (risking 'not another bloody telescope').  While this might match the reality of museum catalogues, would it still allow for a fun game?

The realities of a project I was building in my free time and my lack of graphic design and illustration skills also provided constraints – it had to be browser-based, it couldn't rely on a critical mass of concurrent players to validate actions or content, it had to help the player dive straight into playing and overcome any fears about creating content about museum objects, and it had to use objects ingested through available museum APIs (I selected broad subjects for testing but didn't individually select any objects).

I then added a few extra constraints by deciding to build it as a WordPress plugin – I wanted to take advantage of the CMS-like framework for user logins, navigation and page layout, and I wanted the code I wrote to be usable by others without too much programming overhead.  I'll need to tidy up the code at the end, but once that's done you should be able to install it on any hosted WordPress installation.  I'm making a related plugin to help you populate the database with objects (also part of an experiment in the effectiveness of letting people choose their own subject areas or terms to select playable objects).  I'll talk more about how I worked with those constraints and how they informed the changes I made after evaluation in a later post.

Different games for different purposes
I've been thinking about a museum metadata game typology, which not only considers different types of fun, but also design constraints like:

  • the type and state of the collection (e.g. art works, technical/specialist and social history objects; photographs and other media vs objects; reference collections vs selected highlights; 'tombstone' vs general vs interpretative records)
  • the type of data sought including information curators could add if they had infinite time (detail on the significance of the object, links to other subjects, people, events, objects, collections, etc); information that can be extrapolated from the existing catalogue record; things curators couldn't know (personal history, experiential accounts about the design, manufacture, use, disposal etc of objects); emotional responses; external specialist knowledge; amateur/hobbyist specialist knowledge; synonyms in every day language; terms in other spoken languages

I've also been playing with the idea of linking different game types to different 'life stages' of museum collection metadata.  For example, some games could help a museum work out which of its catalogued items seem more interesting to the public, others help gather tags, create links between items or encourage players to research objects and record new information or links about them, and others still could work well for validating data created in earlier games.  The data I gather through evaluating the games I've designed will help test this model.

So, all that said, if you'd like to play (and help with my evaluation), the two games are:

Donald's detective puzzle – find a fact about an object
Dora's lost data – a simpler tagging game

Survey results: issues facing museum technologists

In August 2010 I asked museum technologists to take a survey designed to help me understand and communicate the challenges faced by other museum technologists (as reported in 'What would you change about your workplace? A survey for museum technologists', and as promised, I'm sharing the results (a little later than intended, but various galleries and my dissertation have been keeping me busy).

There were 79 responses in total, (49 complete responses, the rest were partial).  According to SurveyGizmo's reporting the survey had responses from 10 countries.  The vast majority were from the UK (36%) and the US (49%), possibly reflecting the UK and US focus of the email lists where I publicised the survey.  Respondents were based in a wide range of art, history, science, local authority/government, university and specialist museums (in almost any combination you can think of) and had a variety of roles, including content, technical, project managers and managerial titles.  As reported originally, for the purposes of the survey I defined 'museum technologist' as someone who has expertise and/or significant experience in the museum sector and with the application or development of new technologies.

I've done my own coding work on the results, which I could also share, but I suspect there's more value in the raw results.  I'm also sharing the results to the first two questions as CSV files (compatible with most applications) so you can download and analyse the data: CSV: As a museum technologist, what are the three most frustrating things about your job?, CSV: List any solutions for each of the problems you listed above.  Please note that the data in these files is alphabetised by row, so you should not correlate responses by row number.

My thanks to the people who took the time to respond – I hope there's some value for you in this sampling of the challenges and joys of digital work in museums.  I'd love to hear from you if you use the results, either in a comment or via email.

Question 1: As a museum technologist, what are the three most frustrating things about your job?

First response box:

An institutional culture that values curatorial opinion over the expertise of technologists
Bad management
Becoming impossible to do new work AND maintain existing sites.
Bureaucracy
Central ICT department not being supportive
Colleagues who think of things digital as somehow separate and of lesser importance
Committees
Convincing administration of the value of new technology
Difficulty accessing social networking sites/FTP/etc through Council systems
Funding (lack of)
Going over the same ground again and again
I spend a lot of time doing non-tech work, or helping people with basic IT issues
IT department not implementing effective change management and training.
IT dept walls
IT infrastructure – restrictions and problems
Image rights
Institutional IT provision
Justifying new technologies
Lack of Resources (People)
Lack of clear copyright procedure hampers the greatest ideas
Lack of committment reuslting in long drawn out meetings that never go anywhere
Lack of communication
Lack of decision making from senior management at early stages in the project
Lack of interest in updating technology
Lack of planning
Lack of power to influence major decision making
Lack of resources for web tools/infrastructure
Lack of understanding of what we (as technologists) are trying to achieve
Lack of updated skills in co-workers
Lukewarm funding
Overcoming bureaucracy and overly cautious policy to try new technologies in a timely manner
Pace of sign off
People assuming I know everything about every technology
Senior managment attitudes
Trying to encourage change for the greater good
Unreasonable objectives
Varying age of equipment
Working within IT limitations
Working within existing budgets
bureaucratic oversight
clarity & simplicity of goals
data migration
dfdf
fear of change
getting buy in from people who don't understand the technology
imprecise demands
insufficient staff resources
lack of communication between team members
lack of vision
lengh of time from concept to implementation (it is too long)
mmmm
no $$ for training
not being included early enough in planning processes
not enough time
reactionary IT managers
too many stakeholders and a very conservative attitude to sign off
unrealistic expectations
Getting the management of the museum to take the web seriously and use it themselves to try to understand it
The decentralized culture of our Museum. Each department is doing their own thing, which makes it difficult to access needs, plan for improvements, allocate resources and staff efficiently.
The little understanding colleagues have of the challenges faced (e.g. building a professional website is doable in 1 week with a 300€ budget)
Lack of understanding of digital audiences, trends, issues and technologies by those commissioning digital projects (I call it 'and then it needs a website' syndrome
The organizational structure of the museum. The IT Department should be for networking, desktop support and infrastructure but instead they end up being the ones who call the shots about applications and systems.
Integrating our technologies and ideas into the museum's IT infrastructure e.g. wireless hubs, installing software, updating software etc.

Second response box:

"shiny new toy" syndrom
Assortment of operating systems
Bureaucracy
Changing priorites
Enforcing efficient use of storage space (delete your DUPES!)
Excessive review cycles
Gaining buy-in from overworked staff who need to contribute to tech project
Getting curators to take the web seriously and want to use it
Having other people re-invent things I invented 10 years ago
Institutional IT provision
Institutional blindness to the outside world (i.e., "nobody actually trusts Wikipedia")
Interdepartmental Workflow
Internal "Ownership" of information
Justifying the expense/time of trialling and sharing new ideas
Lack of Finance
Lack of appreciation for the amount of work involved
Lack of funding
Lack of medium/long term visions
Lack of shared museum assets (inter and intra)
Lack of understanding of digital media by senior executives
Lack of understanding of my role at more senior levels and by my peers
Non-existent budgets
Ph.D syndrome.
Some staff negativity about integrating new technologies
Stodgy curators
Tempering desire with reality
Time to just 'play' with new technologies
Too many egos
Too many people involved
Too many tasks seen as top-priority without enough support to get them done.
Understaffed and underfunded
Unwillingness to try small cheap ideas (on the understanding that if they don't work you stop)
Upper management not grasping value of online outreach
Working in isolation
board and execs who are focused on shiny objects, not mission
dealing with the ramifications of technology decisions made by non-technical employees
entrenched views on how things should be done
funding and management structures that lead to short term, siloed thinking
inability to ack quickly and be flexible (cumbesome review process ties up projects)
inablility of coworker to understand projects
institutional resources
lack of staff time or positions alotted to technology (two minds are better than one)
mmm
no say over even how our web page is designed
not enough money
poor instructions
sparse training
tendency for time to get sucked into general office work
unprofessionalism
unreasonable expectations
unwillingness to fund projects
Lack of understanding in the wider museum of the work that we do and the potentials of technologies in learning.
People in museum administration often know less about technologies than their counterpart in the private sector.
Lack of training offered on national scale for those who are beyond beginner level with technology but not an expert
Not having admin rights to my computer and not being allowed to connect my own laptop to the work network
The expectation of a high-impact web presence without making the appropriate content available (in time)
Never knowing what others departments are doing, but still being expected to "fix" whatever when it goes down.
The little commitment others (even people asked/hired to do so) have towards social media, even after tons of workshops.
Turf wars – different staff not working toward a consensus; arguments are recycled and nothing is ever finalized
redundancy–for example, entering metadata for an image from an external source and entering it into our DAM
Funding is spread unevenly. New galleries might come with big pots of money but it's much harder to fund work on existing sites and sections.
Lack of IT understanding by other staff in the museum and in some cases a negative attitude to putting stuff online

Third response box:

"non-profit" pay and no insurance
Always defending my position to condescending curators
Assortment of learning curves among staff
Balancing the demands of day to day tasks with the desire to expand IT use
Bending commercial products to our own needs.
Communication barriers
Conflicting messages about the purpose of online – is it to generate income or provide access?
Cross departmental walls
Cultural stigmatism
Curators/educators living in the dark ages!
Difficulty finding funding/support for less visible tech projects (content architecture, etc.)
Division between web/curatorial/education/etc.
Everyone is scared
Explaining complex systems to co-workers with limited tech background
Getting "sign off"
Hard to sell technology (APIs, etc) to staff who just want their event on the homepage.
Institutional IT provision
Keeping up with web science/standards
Lack of by in by senior management
Lack of change management at institutions
Lack of communication
Lack of professional development
Lack of support
Little allowance to "try out" tech tools/software/web
No time to experiment and try out new things
Projects never finished
Reliance on external consultants
Secret stakeholders appearing late in the production cycle
Software provider lack of focus on end users and Web
That every bit of the organisation has to be involved in every project
Too much dependence on content producers, e.g. curators, gallery authors, education staff
Trying to get other colleagues involved in technology!
Willingness of colleague tech adoption
capacity of organizations to take leaps of faith
dealing with art historians
defining projects in terms of ROI
frequent interruptions during thought-intensive work
lack of adminstrative support in the way of $$
lack of forward planning
misunderstanding of implications
mmm
not enough focus on early prototyping before the tech comes in
not enough staff
not enough staff and too many things to do…
not enough time
passive/aggressive behavior
resistance to new technologies on the basis of their perceived danger/risk
strong aversion to risk-taking, which hampers innovation
supporting software that was incorrectly chose (e.g. retrofitting a CMS to act as a DAMS)
user incompetence
wide range knowledgement needed
Magical thinking about technology: somehow hoping projects will be cheap and cutting edge with few resources devoted to them
There's a web/multimedia team, but all the exhibition design is outsourced, so it's difficult to mount integrated digital projects (that work both online and onsite)
disconnects between depts in larger museums, that make it hard to get all those who could contribute to and benefit from digital projects really engaged
Irrational fear of open source; irrational fears concerning access to collection information and even low-res images.
The fact that doing "online stuff" means you have to solve every problem related to technology ("My iPhone doesn't synch my music, help!")
Convincing staff to use project results (this is true for some staff in key positions. Other staff happy to use the results)
my department uses a DAM system, but others outside my department won't use it but want access to the content archived there

Question 2: List any solutions for each of the problems you listed above

First response box:

$$ for training would be easy to get
Better IT training and also digital awareness training for all staff
Better investment
Better organisational understanding of the importance of project management
Better qualified staff – training
Circumnavigating IT when they sya can't do and supporting it all ourselves.
Cloud based
Creative use of budgets – taking parts from several budgets to make a whole
Education
Fewer and smaller
Focusing on the benefits of the new technology when presenting changes to staff
Good management
Greater funding support for equipment
Hiring further staff
IT managers who are less about security and NO and more about innovation
Improve communication by removing large egos
Keeping to meeting agendas and ensuring people involved are enthusiastic about the project
Long term strategy agreed at top levels to ringfence time and money for non-project based work
Lots of demonstrations
Make responsibilities of depts clearer
Meetings, Meetings, Meetings
More independence from IT
More staff!
Much clearer policy on approach to copyright, possibly by museums supporting one another
New, professionally trained management
Sack the lot of them and start again
Strict procedures and continuously stressing how things work and how they don't.
The acknowledgement at senior levels of competence and experience further down the scale
Training in Project Management
Upgrade technology to a consistent level
Willingness to learn
come up with your own
educate administration, show them how other museums are taking advantage, find funding
fundraising
no foreseeable increase in staffing, so no luck here
none in sight
planning
solutions that we have found or solutions we wish for? The questions is confusing.
steel myself to do it once more in a way that means they can't forget it
umm..if I had a solution I'd be rich :)
Adjust the expectations by explaining the process more in depth and always provide more conservative time estimates, and times that by 150%
We are now submitting a business case to our IT department for us to have access to these sites. Hopefully this will be widened in the future as Council's become more aware of the essential part technology plays in museums.
reallocation of institutional resources to recognise changing technological and social environment
Having highly-placed technologists who are trusted by the museum involved in projects at an early state can help significantly to teach the institution the value of technological expertise.
Advocate your work to anyone who will listen, get involved in projects from the beginning – and try not to let technology lead, only support good ideas
Rethinking contracting policies–especially for Web 2.0 services that are free–and approval processes
Look to private sector technology vendors for workflow and project management techniques and tools or hire consultants (voices from outside are often heard louder than those inside).

Second response box:

$$ we are given we do not always get
Allowing staff to make their own decisions
Cost effective training or events or 'buddying up' to share expertise and experiences
Crossover training
Don't tell, stay away from committees until you have something (good) to show
Encouraging positive comment and activity from outside
Establish an agreed level of autonomy and freedom for web projects
Fix to IT issues that take up so much of my time!
Fundraising specifically for technology as an ongoing need–not just project by project
Involvement of Technologists before design
More educated staff about abilities and weaknesses of technology
More funding and resources for projects
More rewarding work environment
More tech-savvy upper management (happened recently)
More training being offered via bodies such as Museums Galleries Scotland
More trust in teams
New, professionally developed board
Outside normzl dept relationships
Priorities either need coherent justification or to be realigned.
Reassigning permissions
Recruit more staff and do more work in house
Remove large egos
Request more specificity and detail
Speaking to people to explain the complexity and time necessary for project?
Streamlining Project Management
The creation of roles at a senior level with understanding of technology
Training for staff
Trying to get a pot on our web page for e-learning which displays and advocates our work.
agreement on acceptable standards for public facing databases
occasionally half-successful compartmentalization of time spent on specialized and general work
question assumptions
shoot the current managers
sponsorships
strong compromise with staff training
technology being an embedded part of the work, like education
would require a wholesale change in Museum culture – not likely to happen quickly
institution-wide training in Word, PowerPoint, Excel etc AND in newer more interesting tools for presentations (eg Prezi), data visualisation (ManyEyes, Wordle) etc
Increase levels of digital literacy through out organisation and sector by training, workshops and promotion
Write in the importance of technology projects to accomplishing the mission in strategic planning and grant documents and form interdepartmental teams of people to address technology issues and raise technology's profile and comfort level within the institutional culture.
make sure to 'copyright' my own inventions and publicise them before anyone else needs to re-invent them
Show them that colleagues in their field are using the same technology, once they're willing to listen, show how the results will help them, then make participation as easy as possible for them.
If, for every bit of unfounded, unresearched opinion, the technologist can counter with facts about how people actually behave in the world outside the museum, over (large stretches of) time this problem can be gradually allayed.
Presenting the case for how technology can do certain things really well and how it is best find the better fit than to force technology to be what it isn't
Our institution could benefit from professional training on effective communication, but it's not in the budget.
Organising lunches and other team activities to continuously explain and inspire people about new and social media

Third response box:

(Sadly) winning awards
Admin-down promotion of tech initiative adoption
Agreement on stakeholders and sign off processes up front – and sticking to that
Be very strict with project deadlines!
Better communications from the top
Developing a Museum Service strategy for everyone to use IT – like V&A have!
Education
Ensuring that people at senior levels support digital projects
Go and do. Prototype to prove point
Good management
Hired more competent users or remove technically-involved tasks from users
I think we need new ways of demonstrating value other than £s or people through the door
Identify internal skills before commissioning outside consultants
Improved communications – more vision
Informal brown-bag lunches where ideas are pitched and potential explained.
Inventiveness!
Longer timelines, adequate staffing levels
Look for oppurtunity to learn more and implement new systems that help with the day to day work
Make it as easy as possible to use the results
Museums need to start thinking more like libraries
No idea how we can make LA central ICt departments more helpful
Outsource all IT relating to web projects
Professional development for staff
Remove large, scary egos
Smile, help them, and complain in silence.
Some inovative young blood in these roles
Technologists in upper management
Try something small as a pilot to reveal realistic benefits and pitfalls
act of God
bringing techies into the development process earlier in a new exhibit etc.
ditto
effective allocation of scarce resources
rewriting job descriptions to incorporate tech initiatives into everyday tasks
specialization
there is no solution for art historians except possibly to keep them out of museums and galleries
time-shifting certain kinds of work to early morning or evening, outside regular hours
Trying to find public outputs of infrastructure-related technology can help with this problem. The way some museums have begun using collections APIs as, in essence, a PR tool, is a good example of this approach.

Selected responses to Question 3: Any comments on this survey or on the issues raised?

Some comments were about the survey itself (and one comment asked not to be quoted, so I've played it safe and not included it) and didn't seem relevant here.

  • Would like to know what other museum staff feel, but am guessing response may be very similar
  • There is still some trepidation and lack of understanding of what it is exactly that digital technology can play in display, interpretation and education programming. Though there are strong peer networks around digital technology, somehow this doesn't get carried over into further advocacy in the sector in general. In my learning department there is some resistance to the idea of technology being used as a means in itself working across audiences, and it instead has to be tied in to other education officers programmes. The lack of space to experiment and really have some time to develop and explore is also sadly missed as we are understaffed and overstretched.
  • Not enough time, money or staff is true of most museum work, but particularly frustrating when looking at the tools used by the private sector. This imbalance may be part of the source of unreasonable expectations – we've all seen fantastic games and websites and expect that level of quality, but museums have 1/1000th of the budget of a video game studio.
  • The interdepartmental nature of many tech projects has challenged us to define under whose purview these projects should be managed.
  • In my organisation I find the lack of awareness and also lack of desire to do things online difficult to comprehend in this day and age. It is not universal, fortunately the Head of Service gets it but other managers don't. I'm fed up hearing 'if its online they won't visit' and I'm afraid I've given up trying to convince them, instead I tend to just work with the people who can see that putting stuff online can encourage visitors and enhance visits for visitors.
  • Being a federal institution, we receive funds for physical infrastructure, but rarely for technical infrastructure. I would say fear around copyright of digitized collections is a barrier as well.
  • Until the culture of an institution of my size changes at the top, it will continue to be a challenge to get anything through in a timely manner.
  • Funding and resources (staff, time, etc.) are the main roadblock to taking full advantage of the technology that's out there.
  • There needs to be a way to build a proper team within the museum structure and make silos of information available.
  • I think the frustrations I raised are exactly the reason why some of us are in the museum sector – for the challenge.
  • We are fortunate in that we have a very forward-looking Board of Trustees, a visionary CEO and a tech team that truly loves what they do. But we – like any non-profit – are always limited by money and time. We've got loads of great ideas and great talent – we just need the means and the time to be able to bring them to fruition! We have actually rewritten job descriptions to make certain things part of people's everyday workflow and that has helped. Our CEO has also made our technological initiatives (our IVC studios, our online presence, our virtual museum….) part of our strategic plan. So we are extremely fortunate in those respects!
  • I am a content creator, rather than a technie, but as my role is digital, everyone assumes I understand every code language and technological IT issue that there is. And I don't.
  • why is it that those who are not involved in our work have so much to say about how we do our work down to the last detail
  • One of the largest problems faced by IT staff in museums is the need to push the envelop of technology while working within very limited budgets. There is always a desire to build the newest and best, but a reluctance to staff and budget for the upkeep and eventual use and maintenance of the new systems. That said, working for a museum environment offers more variety and interesting projects than any for-profit job could ever provide.

What would you change about your workplace? A survey for museum technologists

[Update: I've shared the data]

This week I launched a survey designed to help me understand and communicate the challenges faced by other museum technologists.

It's research for a chapter in a forthcoming book on museums on the web and social media in the first instance, but I'd left the terms and conditions fairly open as I wanted to be able to share and/or re-use the data in future – I wasn't sure if this would put people off, but I figured it was better to be upfront than to end up with great data I couldn't share.

Someone wrote to me to ask what the questions were – they didn't feel qualified to take it themselves but couldn't see all the questions without starting the survey.  I figure it'll also help with the bounce rate if I share them, so here you go:

1. As a museum technologist, what are the three most frustrating things about your job?

For this survey, I'm defining 'museum technologist' as someone who has expertise and/or significant experience in the museum sector and with the application or development of new technologies.

2. List any solutions for each of the problems you listed above

3. Any comments on this survey or on the issues raised?

4. What's your main job role? (if you don't mind it potentially being quoted)

5. Please enter your institution name and/or type (e.g. art gallery, history museum, local authority museum, science centre). (if you don't mind it potentially being quoted)

It's pretty simple – 'what are the three most frustrating things about your job' is the main question, the rest are aimed at providing just enough additional information to provide pointers to the effects of different factors. I didn't want to ask people for so much information that they'd be identifiable as I felt that might make people hold back.  I thought about saying 'things other than lack of resources/time/money' as they're pretty much a given and they're not unique to the museum sector, but I figured they're also too important too ignore.
For further context, when I posted it to the MCG and MCN lists I said:

I'm particularly interested in opportunities and problems that arise when (new) technologies meet (old) museums. … Your answers will help build a body of evidence that could help make a case for improvements in the way museums understand the issues and expertise around using technology to engage audiences, or at least help us understand what the solutions might be. And at the very least you get to vent a bit!

I'm running the survey until August 31 and my initial analysis will be completed by mid-September.  If you'd like to take the survey, or know someone who should, the address is http://www.surveygizmo.com/s3/348155/Challenges-facing-museum-technologists (or http://bit.ly/95oGtr if shorter is easier).

Finally, thanks to the person who suggested making the text boxes wider – I've done that now.

'Game mechanics for social good: a case study on interaction models for crowdsourcing museum collections enhancement'

I've been very quiet lately – exams for my MSc and work on the digital infrastructure for two new galleries (and a contemporary science news website) opening next week at the Science Museum have kept me busy – but I wanted to take a moment to post about my dissertation project. (Which reminds me, I should write up the architecture I designed to extend our core Sitecore CMS with WordPress to support social media-style interactions with Science Museum-authored content.)

Anyway. This project is for my dissertation for City University's Human-Centred Systems MSc. I'm happy to share the whole outline, but it's a bit academic in format for a blog post so I've just posted an excerpt here. I'd love to hear your comments, particularly if you know of or have been involved in creating, crowdsourced museum projects or games for social good.

'Game mechanics for social good: a case study on interaction models for crowdsourcing museum collections enhancement' is the current title – it's a bit of a mouthful but hopefully the project will do what it says on the tin.

Project description
The primary focus of this project is the design and evaluation of interactions applied to the context of an online museum collection in order to encourage members of the public to undertake specific tasks that will help improve the website.

The project will include a design and build component to create game-like interfaces for testing and evaluation, but the main research output is the analysis of museum crowdsourced projects and 'games for social good' to develop potential models for game-like interactions suitable for museum collections, and the subsequent evaluation of the proposed interaction models.

Aims and Objectives
This project aims to answer this question: can game-like interactions be designed to motivate people to undertake tasks on museum websites that will improve the overall quality of the website for other visitors?

More specifically, which elements of game mechanics are effective when applied to interfaces to crowdsource museum collections enhancement?

Objectives

  • Design game-like interaction models applicable to cultural heritage content and audiences through research, analysis and creativity workshops
  • Build an application and interfaces to create and store user-created content linked to collections content
  • Evaluate the effectiveness of game-like interaction models for eliciting useful content

Theory
Recent projects such as Armchair Revolutionary[1] and earlier projects such as Carnegie Mellon University's 'Games with a purpose'[2] and InterroBang?![3] are indicative of the trend for 'games for social good'. Crowdsourced projects such as the Guardian newspapers examination of MPs expense claims[4], the V&A Museum's image cropping[5], Brooklyn Museum's tagging game[6], the National Library of Australia's collaborative OCR corrections[7]; Chen's (2006) study of the application of Csikszentmihalyi's theory of 'flow' to game design; and Dr Jane McGonigal's ideas about multiplayer games as 'happiness engines'[8] all suggest that 'playful interactions' and crowd participation could be applied to help create specific content improvements on museum sites. Game mechanisms may help make tasks that would not traditionally considered fun or relaxing into a compelling experience.

Within the terms of this project, the output of a game-like interaction must produce an effect outside the interaction itself – that is, the result of a user's interactions with the site should produce beneficial effects for other site visitors who are not involved in the original interactions. To achieve this, it must generate content to enhance the site for subsequent visitors. Methods to achieve this could include creating trails of related objects, entering tags to describe objects, writing alternative labels or researching objects – these will be defined during the research phase and creativity workshops.

Methods and tools

The project is divided into several stages, each with their own methodology and considerations.

Research
The preliminary research process involves a literature review, research into game mechanics and the theory of flow, and research into museum audiences online. It will also include a series of short semi-structured interviews with people involved in creating crowdsourced projects on museum sites or game-like interactions to encourage the completion of set tasks (e.g. games for social good) in order to learn from their reflections on the design process; and analysis of existing sites in both these areas against the theories of game design. This research will define the metrics of the evaluation phase.

Creativity workshop(s)
The results of this research phase sets the parameters for creativity workshops designed to come up with ideas and possible designs for the game-like interfaces to be built. Possible objectives for the creativity workshop include:

  • designing methods for building different levels of challenge into the user experience in an environment that does not easily support different levels of challenge when museum-related skills remain at a constant level
  • creating experiences that are intrinsically rewarding to enable 'flow' within the constraints of available content

Build and test
In turn, the creativity workshops will help determine the interfaces to be built and tested in the later part of the project. The build will be iterative, and is planned to involve as many build-test-review-build iterations as will fit in the allocated time, in order to test as many variant interaction models as possible and support optimisation of existing designs after evaluation. User recruitment in this phase may be a sample of convenience from the target age group.

The interfaces will be developed in HTML, CSS and JavaScript, and published on a WordPress platform. This allows a neat separation of functionality and interface design. Session data (date, interface version, tester ID) can be recorded alongside user data. WordPress's template and plug-in based architecture also supports clear versioning between different iterations of the design, allowing reconstruction of earlier versions of the interfaces for later comparison, and enabling possible split A/B trials.

Analysis and write-up
Analysis will include the results of user testing and user data recorded in the WordPress platform to evaluate the performance of various interface and interaction designs. If the platform attracts usage outside the user testing sessions it may also include log file or Google Analytics analysis of use of the interfaces.

[1] https://www.armrev.org
[2] http://www.gwap.com/
[3] http://www.playinterrobang.com/
[4] http://mps-expenses.guardian.co.uk/
[5] http://collections.vam.ac.uk/crowdsourcing/
[6] http://www.brooklynmuseum.org/opencollection/tag_game/start.php
[7] http://newspapers.nla.gov.au/ndp/del/home
[8] http://www.futureofmuseums.org/events/lecture/mcgonigal.cfm

Join in the conversation about Wikimedia @ MW2010

Wikimedia@MW2010 is a workshop to be held in Denver in April, just before the Museums and the Web 2010 conference.  The goal is to develop 'policies that will enable museums to better contribute to and use Wikipedia or Wikimedia Commons, and for the Wikimedia community to benefit from the expertise in museums'.

If you've got stuff you want to say, you can dive right into the conversation – there's a whole bunch of conversations at http://conference.archimuse.com/forums/wikimediamw2010, including 'Legal and Business Model Barriers to Collaboration, 'Notability Criteria' and 'Metrics for Museums on Wikipedia'.

I'm going to be at the workshop and will do my best to represent any issues raised at the meeting.  I think it's particularly important that we avoid 'Feeling glum after GLAM-WIKI' if we possibly can, so I'd like to go there with a really good understanding of the possible points of resistance, clashes in organisational culture or world view, incompatible requirements or wishlists so that they can be raised and hopefully dealt with during the in-person workshop.  I'd love to hear from you if there are messages you want to pass on.

I'm also thinking about an informal meetup in London to help cultural heritage people articulate some of the issues that might help or hinder collaboration so they can be represented at the workshop – if you're a museum, gallery, archive, library or general cultural heritage bod, would that be useful for you?

Survey results: is it friendly or weird when a museum twitter account follows you back?

Last Tuesday, I asked 'If you follow a museum on twitter, is it friendly or weird if it follows you back?' after calls from some quarters for #followavisitor #followamember or #MuseumsFollowYouBack days after followamuseum day on Twitter.  The poll gathered 50 responses overall and I've presented an overview of the results here.

Question 2 was added in response to a suggestion from a respondent after 20 responses had already been given, so for this reason alone, the results should not be taken as anything other than an interesting indication of responses.  I've shared the written responses to various questions, and provided a quick and dirty analysis of the results.

1. If you follow a museum on twitter, do you want it to follow you back?

Yes 49%
No 26.50%
It depends 26.50%

13 further comments were given for 'it depends':

  • if they're conversational or broadcasting
  • I hope they do, they don't have to.
  • Depends on what the account is doing. If it's just sending out announcements, who cares if it follows you back? If they're actually using Twitter, and there's an actual person back there somewhere doing something interesting, I'd be pleased if they decided to follow me, like any other user.
  • Of couirse I'd like it, but I understand if they don't due to over-following capacity!
  • If the museum is going to engage w/ me then yes; if it's just to broadcast I'm on the fence
  • I'm an art historian, so if an art museum started to follow me, I would be flattered! But if another kind of museum followed me, I would be slightly confused. So I think it would depend entirely on the profession of the person and if they use their account in a professional way
  • If they start wanting to be my best bud, I'd probably get creeped out and block them.
  • I wouldn't mind being followed, but not as a data point in a marketing database or to get impersonal spam.
  • I don't think I really have a strong view either way.
  • Why?
  • If I've started a discussion with said museum through twitter
  • Don't mind either way in most cases
  • it has no material effect — I don't gain anything from it following me.
I also posted the question on Facebook, and two people said it was weird. One went further, "I think it's weird, unless you primarily tweet about museums. I assume that anyone following me that is following more than 200 people doesn't actually read my tweets.".

2. If you follow a museum on twitter, do you mind if it follows you back?

Yes 2%
No 44%
It depends 14%
Skipped 40%
[See note above about the number of 'skipped' responses]

7 further comments were given for 'it depends':

  • I'd rather be able to look at who you follow to find other twitterers of interest. Can't do that if you follow thousands of people back. Be selective so we can look thru them.
  • not unless my tweet is museum related
  • It depends on whether I know who is behind the tweets. Being a museum professional, sometimes they are colleagues, and that's okay with me.
  • I don't really care, but I think it's silly.
  • I just don't see why they would, it doesn't help either of us
  • See above :)
  • I would prefer it to follow back, especially if it's relevant to my own areas of historical interest, but no one has to follow anyone they don't want to.

So it looks like you can't win – almost 50% of new followers expect you to follow them and 50% either don't, or only do under some circumstances.  As you can see from the responses to questions 3 and 4 (below), the results have presumably been skewed as 50% of respondents have a close involvement with museums, and a whopping two-thirds have a professional or academic interest in social media. I'm using the free version of SurveyMonkey so can't easily split out the 'social media' or 'museum professional' responses from the rest to see if people who are neither have different views on reciprocal following.

The only way to get a sense of whether followers of your particular museum account expect to be followed back, or mind being followed back, may be to ask them directly.

Of the people who were able to answer question 2, a very small minority unequivocally minded being followed by a museum account, but 44% of those who answered don't mind if a museum account doesn't follow them back.

Another interesting question would have been 'is it friendly or weird if an organisation follows you after you mention them?' – if you do any more research into the issue, let me know.

Question 6 asked for 'Any other comments?'.

  • Though I don't work in a museum, I work in or with museums. I think the main issue is that museums tweeting should have personality, you should feel it's a person (or group of people) that want to engage with you. I think if they follow me, it's more likely they will hear me and engage.
  • By following and being followed by a museum it creates a sense of community (although of course I realize the museums won't have time to read all the tweets).
  • I run a twitter account on behalf of a library, and think it's good manners to follow back someone who follows you (like returning a hello). I always try to reply to people who want to talk to us, but try not to butt into conversations that are *about* us, but which don't want us to reply to them (this can be tricky). As a user, by and large – I only want to talk to museums when I know the people doing the tweeting – and in New Zealand, I know most of these people anyway. Internationally, I do the same thing. Courtney Johnston, @auchmill, @nlnz
  • I like when museums respond to my comments or @ replies, but I'm not as comfortable with them following me. Being responsive is different than following.
  • Twiter has become one of the best sources for professional info & contacts @innova2
  • It would be more useful for the museum to keep track of hashtags, etc.
  • I want to communicate with people I follow, so following me back makes it easier. I don't expect them to read all/most of what I say, but it's nice…especially on my protected account (otherwise they never see the @)
  • Museums rock!
  • I don't really mind either way. I find it flattering if an institution wants to follow me. They must think I have something to say!
  • I feel that museums have a great opp to get more individuals involved in history and the arts via social media – personally I follow a few and am always pleased to hear about new exhibits, events, etc.
  • I work with museums, that's why I would like them to follow me back. I use Twitter for work, so I don't mind if they follow back. If I would talk to my friends over Twitter about private stuff maybe I WOULD mind….
  • If a museum (or anyone) didn't follow back, I would probably unfollow after a while unless their tweets were really something special.
  • I want people/institutions to follow me if they have genuine interest in my tweets – the same criteria I apply when choosing who to follow myself!
  • The real question probably should have been (or maybe an additional question should have been): do you MIND if a museum you follow follows you back. Because really, I don't necessarily want them to, but I don't mind if they do. I have the power to block if need be.
  • I think if the Museum was clear about WHY it was following me on twitter it would be less "stalkerish". In general I somewhat expect to be followed by those I am following. Although I am not sure if organizations (Museums) really need to follow individuals. I would imagine the Museum's staff would be overwhelmed with the number of completely unrelated tweets. What would be the advantage that couldn't be obtained better by simply searching twitter for key terms related to the museum, content, exhibit, etc? (Note: I do not work "in" a museum but have worked with over 6 museums to define and develop their websites and web marketing activities)
  • I manage a twitter feed for a project at a science center. I follow people, organizations and businesses that are in the service area for the project (a watershed). I also follow other organizations that are working on similar issues (water quality).
  • I think not following people back is poor Twitter etiquette. That is like saying to someone, "Listen to me! But I won't listen to you!"
  • 1. Personally, I find follow bots mildly more insulting than not being followed back. 2. In my professional capacity tweeting for a museum, if someone @mentions us, I follow them (I consider it friendly). The fact that this could be done equally well (and more efficiently) by a bot disturbs me a bit.
  • I tweet with an interest in culture, art, and museums in mind. It's more of a compliment for museums to follow back than a feeling of being stalked, as it shows interest in its reader's tweets.
  • Leave the poor souls alone, they only want to know what's going on at your museum. I've blocked Museum of London (and I have every reason to trust them. Or not)
  • How is it stalkerish…dumb survey
  • Happy to be followed if it would help the museum understand more about its audience  
So that's that.   I thought 'being responsive is different than following' summed things up quite nicely, but whatever your view, some interesting opinions have been expressed above.
I hadn't considered before that not following someone back was rude – I must appear terribly rude on my personal accounts but I just can't keep up with so many accounts, especially as I can have lots of time away from the keyboard.

Finally, the demographic questions (kept brief to keep the survey short)
3. Do you work, study or volunteer in a museum?
Yes 56%
No 44%

4. Do you work, study or volunteer in social media?
Yes 68%
No 32%

5. Where in the world are you?

Countries Total respondents
Australia 2
Belgium 1
Canada 2
Germany 1
Netherlands 4
New Zealand 1
Norway 1
Spain 2
Switzerland 1
UK 17
USA 18