'…and they all turn on their computers and say 'yay!" (aka, 'mapping for humanists')

I'm spending a few hours of my Sunday experimenting with 'mapping for humanists' with an art historian friend, Hannah Williams (@_hannahwill).  We're going to have a go at solving some issues she has encountered when geo-coding addresses in 17th and 18th Century Paris, and we'll post as we go to record the process and hopefully share some useful reflections on what we found as we tried different tools.

We started by working out what issues we wanted to address.  After some discussion we boiled it down to two basic goals: a) to geo-reference historical maps so they can be used to geo-locate addresses and b) to generate maps dynamically from list of addresses. This also means dealing with copyright and licensing issues along the way and thinking about how geospatial tools might fit into the everyday working practices of a historian.  (i.e. while a tool like Google Refine can generate easily generate maps, is it usable for people who are more comfortable with Word than relying on cloud-based services like Google Docs?  And if copyright is a concern, is it as easy to put points on an OpenStreetMap as on a Google Map?)

Like many historians, Hannah's use of maps fell into two main areas: maps as illustrations, and maps as analytic tools.  Maps used for illustrations (e.g. in publications) are ideally copyright-free, or can at least be used as illustrative screenshots.  Interactivity is a lower priority for now as the dataset would be private until the scholarly publication is complete (owing to concerns about the lack of an established etiquette and format for citation and credit for online projects).

Maps used for analysis would ideally support layers of geo-referenced historic maps on top of modern map services, allowing historic addresses to be visually located via contemporaneous maps and geo-located via the link to the modern map.  Hannah has been experimenting with finding location data via old maps of Paris in Hypercities, but manually locating 18th Century streets on historic maps then matching those locations to modern maps is time-consuming and she suspects there are more efficient ways to map old addresses onto modern Paris.

Based on my research interviews with historians and my own experience as a programmer, I'd also like to help humanists generate maps directly from structured data (and ideally to store their data in user-friendly tools so that it's as easy to re-use as it is to create and edit).  I'm not sure if it's possible to do this from existing tools or whether they'd always need an export step, so one of my questions is whether there are easy ways to get records stored in something like Word or Excel into an online tool and create maps from there.  Some other issues historians face in using mapping include: imprecise locations (e.g. street names without house numbers); potential changes in street layouts between historic and modern maps; incomplete datasets; using markers to visually differentiate types of information on maps; and retaining descriptive location data and other contextual information.

Because the challenge is to help the average humanist, I've assumed we should stay away from software that needs to be installed on a server, so to start with we're trying some of the web-based geo-referencing tools listed at http://help.oldmapsonline.org/georeference.

Geo-referencing tools for non-technical people

The first bump in the road was finding maps that are re-usable in technical and licensing terms so that we could link or upload them to the web tools listed at http://help.oldmapsonline.org/georeference.  We've fudged it for now by using a screenshot to try out the tools, but it's not exactly a sustainable solution.  
Hannah's been trying georeferencer.org, Hypercities and Heurist (thanks to Lise Summers ‏@morethangrass on twitter) and has written up her findings at Hacking Historical Maps… or trying to.  Thanks also to Alex Butterworth @AlxButterworth and Joseph Reeves @iknowjoseph for suggestions during the day.

Yahoo! Mapmixer's page was a 404 – I couldn't find any reference to the service being closed, but I also couldn't find a current link for it.

Next I tried Metacarter Labs' Map Rectifier.  Any maps uploaded to this service are publicly visible, though the site says this does 'not grant a copyright license to other users', '[t]here is no expectation of privacy or protection of data', which may be a concern for academics negotiating the line between openness and protecting work-in-progress or anyone dealing with sensitive data.  Many of the historians I've interviewed for my PhD research feel that some sense of control over who can view and use their data is important, though the reasons why and how this is manifested vary.

Screenshot from http://labs.metacarta.com/rectifier/rectify/7192


The site has clear instructions – 'double click on the source map… Double click on the right side to associate that point with the reference map' but the search within the right-hand side 'source map' didn't work and manually navigating to Paris, then the right section of Paris was a huge pain.  Neither of the base maps seemed to have labels, so finding the right location at the right level of zoom was too hard and eventually I gave up.  Maybe the service isn't meant to deal with that level of zoom?  We were using a very small section of map for our trials.

Inspired by Metacarta's Map Rectifier, Map Warper was written with OpenStreetMap in mind, which immediately helps us get closer to the goal of images usable in publications.  Map Warper is also used by the New York Public Library, which described it as a 'tool for digitally aligning ("rectifying") historical maps … to match today's precise maps'.  Map Warper also makes all uploaded maps public: 'By uploading images to the website, you agree that you have permission to do so, and accept that anyone else can potentially view and use them, including changing control points', but also offers 'Map visibility' options 'Public(default)' and 'Don't list the map (only you can see it)'.

Screenshot showing 'warped' historical map overlaid on OpenStreetMap at http://mapwarper.net/

Once a map is uploaded, it zooms to a 'best guess' location, presumably based on the information you provided when uploading the image.  It's a powerful tool, though I suspect it works better with larger images with more room for error.  Some of the functionality is a little obscure to the casual user – for example, the 'Rectify' view tells me '[t]his map either is not currently masked. Do you want to add or edit a mask now?' without explaining what a mask is.  However, I can live with some roughness around the edges because once you've warped your map (i.e. aligned it with a modern map), there's a handy link on the Export tab, 'View KML in Google Maps' that takes you to your map overlaid on a modern map.  Success!

Sadly not all the export options seem to be complete (they weren't working on my map, anyway) so I couldn't work out if there was a non-geek friendly way to open the map in OpenStreetMap.

We have to stop here for now, but at this point we've met one of the goals – to geo-reference historical maps so locations from the past can be found in the present, but the other will have to wait for another day.  (But I'd probably start with openheatmap.com when we tackle it again.  Any other suggestions would be gratefully received!)

(The title quote is something I heard one non-geek friend say to another to explain what geeks get up to at hackdays. We called our experiment a 'hackday' because we were curious to see whether the format of a hackday – working to meet a challenge within set parameters within a short period of time – would work for other types of projects. While this ended up being almost an 'anti-hack', because I didn't want to write code unless we came across a need for a generic tool, the format worked quite well for getting us to concentrate solidly on a small set of problems for an afternoon.)

Notes from a preview of the updated Historypin

The tl;dr version: inspiring project, great enhancements; yay!

Longer version: last night I went to the offices of We Are What We Do for a preview of the new version of HistoryPin. Nick Poole has already written up his notes, so I'm just supplementing them with my own notes from the event (and a bit from conversations with people there and the reading I'd already done for my PhD).

Screenshot with photo near WAWWD office (current site)

Historypin is about bridging the intergenerational divide, about mass participation and access to history, about creating social capital in neighbourhoods, conserving and opening up global archival resources (at this stage that's photographs, not other types of records).  There's a focus on events and activities in local communities. [It'd be great to get kids to do quick oral history interviews as they worked with older people, though I think they're doing something like it already.]

New features will include a lovely augmented reality-style view in streetview; the ability to upload and explore video as well as images; a focus on telling stories – 'tours' let you bring a series of photos together into a narrative (the example was 'the arches of New York', most of which don't exist anymore).  You can also create 'collections', which will be useful for institutions.  They'll also be available in the mobile apps (and yes, I did ask about the possibility of working with the TourML spec for mobile tours).

The mobile apps let you explore your location, explore the map and contribute directly from your phone.  You can use the augmented reality view to overlap old photos onto your camera view so that you can take a modern version of an old photo. This means they can crowdsource better modern images than those available in streetview as well as getting indoors shots.  This could be a great treasure hunt activity for local communities or tourists.  You can also explore collections (as slideshows?) in the app.

They're looking to work with more museums and archives and have been working on a community history project with Reading Museum.  Their focus on inclusion is inspiring, and I'll be interested to see how they work to get those images out into the community.  While there are quite a few 'then and now' projects focused on geo-locating old images around I think that just shows that it's an accessible way of helping people make connections between their lives and those in the past.

A quick correction to Nick's comments – the Historypin API doesn't exist yet, so if you have ideas for what it should do, it's probably a good time to get in touch.  I'll be thinking hard about how it all relates to my PhD, especially if they're making some of the functionality available.

My PhD proposal (Provisional title: Participatory digitisation of spatially indexed historical data)

[Update: I'm working on a shorter version with fewer long words. Something like crowdsourcing geolocated historial materials/artefacts with specialist users/academic contributors/citizen historians.]

A few people have asked me about my PhD* topic, and while I was going to wait until I'd started and had a chance to review it in light of the things I'm already starting to learn about what else is going on in the field, I figured I should take advantage of having some pre-written material to cover the gap in blogging while I try to finish various things (like, um, my MSc dissertation) that were hijacked by a broken wrist. So, to keep you entertained in the meantime, here it is.

Please bear in mind that it's already out-of-date in terms of my thinking and sense of what's already happening in the field – I'm really looking forward to diving into it but my plan to spend some time thinking about the project before I started has been derailed by what felt like a year of having an arm in a cast.

* I never got around to posting about this because my disastrous slip on the ice happened just two days after I resigned, but I'm leaving my job at the Science Museum to take up the offer of a full-time PhD in Digital Humanities at the Open University in mid-March.

Provisional title: Participatory digitisation of spatially indexed historical data

This project aims to investigate 'participatory digitisation' models for geo-located historical material.

This project begins with the assumption that researchers are already digitising and geo-locating materials and asks whether it is possible to create systems to capture and share this data. Could the digital records and knowledge generated when researchers access primary materials be captured at the point of creation and published for future re-use? Could the links between materials, and between materials and locations, created when researchers use aggregated or mass-digitised resources, be 'mined' for re-use?

Through the use of a case study based around discovering, collating, transforming and publishing geo-located resources related to early scientific women, the project aims to discover:

  • how geo-located materials are currently used and understood by researchers,
  • what types of tools can be designed to encourage researchers to share records digitised for their own personal use
  • whether tools can be designed to allow non-geospatial specialists to accurately record and discover geo-spatial references
  • the viability of using online geo-coding and text mining services on existing digitised resources

Possible outcomes include an evaluation of spatially-oriented approaches to digital heritage resource discovery and use; mental models of geographical concepts in relation to different types of historical material and research methods; contributions to research on crowdsourcing digital heritage resources (particularly the tensions between competition and co-operation, between the urge to hoard or share resources) and prototype interfaces or applications based on the case study.

The project also provides opportunities to reflect on what it means to generate as well as consume digital data in the course of research, and on the changes digital opportunities have created for the arts and humanities researcher.

** This case study is informed by my thinking around the possibilities of re-populating the landscape with references to the lives, events, objects, etc, held by museums and other cultural heritage institutions, e.g. outside museum walls and by an experimental, collaborative project around 'modern bluestockings', that aimed to locate and re-display the forgotten stories around unconventional and pioneering women in science, technology and academia.

Getting closer to a time machine-x-curator in your pocket

If life is what happens to you while you're busy making other plans thenI'm glad I've been busy planning various things, because it meant that news of the EU-funded iTacitus project was a pleasant surprise. The project looked at augmented reality, itinerary planning and contextual information for cultural tourism.

As described on their site and this gizmowatch article, it's excitingly close to the kind of 'Dopplr for cultural heritage' or 'pocket curatr' I've written about before:

Visitors to historic cities provide the iTacitus system with their personal preferences – a love of opera or an interest in Roman history, for example – and the platform automatically suggests places to visit and informs them of events currently taking place. The smart itinerary application ensures that tourists get the most out of each day, dynamically helping them schedule visits and directing them between sites.
Once at their destination, be it an archaeological site, museum or famous city street, the AR component helps bring the cultural and historic significance to life by downloading suitable AR content from a central server.

There's a video showing some of the AR stuff (superimposed environments, annotated Landscapes) in action on the project site. It didn't appear to have sound so I don't know if it also demonstrated the 'Spatial Acoustic Overlays'.

The location-aware future is here (and why cities suck but are good)

Thought-provoking article in Wired on the implications of location-aware devices for our social relationships, privacy concerns, and how we consume and publish geo-located content:

I Am Here: One Man's Experiment With the Location-Aware Lifestyle

The location-aware future—good, bad, and sleazy—is here. Thanks to the iPhone 3G and, to a lesser extent, Google's Android phone, millions of people are now walking around with a gizmo in their pocket that not only knows where they are but also plugs into the Internet to share that info, merge it with online databases, and find out what—and who—is in the immediate vicinity. That old saw about how someday you'll walk past a Starbucks and your phone will receive a digital coupon for half off on a Frappuccino? Yeah, that can happen now.

Simply put, location changes everything. This one input—our coordinates—has the potential to change all the outputs. Where we shop, who we talk to, what we read, what we search for, where we go—they all change once we merge location and the Web.

The article neatly finishes with a sense of 'the more things change, the more things stay the same', which seems to be one of the markers of the moments when technologies are integrated into our lives:

I had gained better location awareness but was losing my sense of place. Sure, with the proper social filters, location awareness needn't be invasive or creepy. But it can be isolating. Even as we gradually digitize our environment, we should remember to look around the old-fashioned way.

Found via Exporting the past into the future, or, "The Possibility Jelly lives on the hypersurface of the present" which in turn came via a tweet.

I also recently enjoyed 'How the city hurts your brain… and what you can do about it'. It's worth learning how you can alleviate the worst symptoms, because it seems cities are worth putting up with:

Recent research by scientists at the Santa Fe Institute used a set of complex mathematical algorithms to demonstrate that the very same urban features that trigger lapses in attention and memory — the crowded streets, the crushing density of people — also correlate with measures of innovation, as strangers interact with one another in unpredictable ways. It is the "concentration of social interactions" that is largely responsible for urban creativity, according to the scientists.

Portable mapping applications make managers happy

This webmonkey article, Multi-map with Mapstraction, about an 'open source abstracted JavaScript mapping library' called Mapstraction is perfectly on target for organisations that worry about relying on one mapping provider.

How many of these have you heard as possible concerns about using a particular mapping service?

  • Current provider might change the terms of service
  • Your map could become too popular and use up too many map views
  • Current provider quality might get worse, or they might put ads on your map
  • New provider might have prettier maps
  • You might get bored of current provider, or come up with a reason that makes sense to you

They're all reasonable concerns. But look what the lovely geeks have made:

The promise of Mapstraction is to only have to change two lines of code. Imagine if you had a large map with many markers and other features. It could take a lot of work to manually convert the map code from one provider to another.

And functionality is being expanded. I liked this:

One of my favorite Mapstraction features is automatic centering and zooming. When called on a map with multiple markers, Mapstraction calculates the center point of all markers and the smallest zoom level that will contain all the markers.

Open source rocks! Not only can you grab the code and have someone maintain it for you if you ever need to, but it sounds like a labour of geek love:

Mapstraction is maintained by a group of geocode lovers who want to give developers options when creating maps.

'Finding yourself with Fire Eagle' at WSG Findability

On Wednesday I went to the WSG London Findability event, and I've finally got the last of my notes up.

The final talk was from Steve Marshall, on 'Finding yourself with Fire Eagle'.

Steve doesn't work on Fire Eagle but made the Python library.

Fire Eagle is a service that helps you manage your location data.

Most location-aware applications have two parts – getting the location, and using the location.

Better model – distribute the location information, but the application getting the location still has to know who's using it.

Even better model: a brokering service. Fire Eagle sits between any 'getting' applications and any 'using' applications, and handles the exchange.

[FWIW, 'Fire Eagle is a brokering service for location data' is probably the best explanation I've heard, and I'd heard it before but I needed the context of the 'get' and the 'use' applications it sits between for it to stick in my brain.]

So how does it work? In the web application context (it's different for desktop or mobile applications):
Web app: app asks for Request Token
Fire Eagle: returns Request Token
Web app: user sent to Fire Eagle with token in URL
Fire Eagle: user chooses privacy levels and authorises app
Web app: user sent back to callback URL with Request Token
Web app: app initiates exchange of Request Token
Fire Eagle: Request Token exchanged for Access Token
Web app: app stores Access Token for user

You can manage your applications, and can revoke permissions (who can set or get your location) at any time. You can also temporarily hide your location, or purge all your data from the service. [Though it might be kept by the linked applications.]

How to use:
1. Get API key
2. Authenticate with user (OAuth)
3. Make API call

Concepts:
Locations can be a point or a bounding box.
Location hierarchy – a set of locations at varying degrees of precision.

[There was some good stuff on who could/is using it, and other ideas, but my notes got a bit useless around this point.]

In summary: you can share your location online, control your data and privacy, and easily build location services.

Discussion:
Question: what makes it special? Answer: it's not coupled to anything. It plays to the strengths of the developers who use it.

'Fire Eagle: twitter + upcoming + pixie dust'.

URLs are bookmarkable, which means they can be easy to use on phone [hooray]. It doesn't (currently) store location history, that's being discussed.

Qu: what's the business model? Ans: it's a Brickhouse project (from an incubator/start-up environment).

All methods are http requests at the moment, they might also use XMP ping.

Qu: opening up the beta? Ans: will give Fire Eagle invitations if you have an application that needs testing.

I had to leave before the end of the questions because the event was running over time and I had to meet people in another pub so I missed out on the probably really interesting conversations down the pub afterwards.

My notes:
Looking at their hierarchy of 'how precisely will you let application x locate you', it strikes me that it's quite country-dependent, as a postcode identifies a location very precisely within the UK (to within one of a few houses in a row) while in Australia, it just gives you the area of a huge suburb. I'm not sure if it's less precise in the US, where postcodes might fit better in the hierarchy.

I've also blogged some random thoughts on how services like Fire Eagle make location-linked cultural heritage projects more feasible.

Questions from 'Beyond Single Repositories' at MW2008

I'm still working on getting my notes from Museums and the Web in Montreal online.

These are notes from the questions at the 'Beyond Single Repositories' session. This session was led by Ross Parry, and included the papers Learning from the People: Traditional Knowledge and Educational Standards by Daniel Elias and James Forrest and The Commons on Flickr: A Primer by George Oates.

This clashed with the User-Generated Content session that I felt I should see for work, but I managed to sneak in at the end of Ross's session. I expected this room to be packed, but it wasn't. I guess the ripples of user-generated content and Web 2.0-ish stuff are still spreading beyond the geeks, and the pebbles of single repositories and the semantic web have barely dropped into the pond for most people. As usual, all mistakes are mine – if you asked a question and I haven't named you or got your question wrong, drop me a line.

Quite a lot of the questions related to 'The Commons'.

There was a question about the difference between users who download and retain context of images, versus those who just download the image and lose all context, attribution, etc. George: Flickr considered putting the metadata into EXIF but it was problematic and wasn't robust enough to be useful.

Another question: how to link back to institution from Flickr? George: 'there's this great invention called the hyperlink'. And links can also go to picture libraries to buy prints.

[I need to check this but it could really help make the case for Commons in museums if that's the case. We might also be able to target different audiences with different requirements – e.g. commercial publications vs school assignments. I also need to check if Flickr URLs are permanent and stable.]

Seb Chan asked: how does business model of having images on Flickr co-exist with existing practices?

Flickr are cool with museums putting in content at different resolutions – it's up to institution to decide.

"It's so easy to do things the correct way" so please teach everyone to use CC licence stuff appropriately.

Issues are starting to be raised about revenue sharing models.

[I wonder if we could put in FOI requests to find out exactly how much revenue UK museums make from selling images compared to the overhead in servicing commercial picture libraries, and whether it varies by type of image or use. It'd be great if we could put some Museum of London/MoLAS images on Commons, particularly if we could use tagging to generate multilingual labels and re-assess images in terms of diversity – such an important issue for our London audiences; or to get more images/objects geo-located. I also wonder if there are any resourcing issues for moderation requirements, or do we just cope with whatever tags are added?]

Update: following the conference, Frankie Roberto started a discussion on the Museums Computer Group list under the subject 'copyright licensing and museums'. You have to be a member to post but a range of perspectives and expertise would really help move this discussion on.

Browse with maps on Flickr

Flickr have introduce a new 'places' feature, which makes geo-tagged photos easier to find by navigating through a map, browsing or searching. There's an end-user focussed screencast explaining how it works. There are more technical links under the 'Are you nerdy?' heading.

Features like this and Google maps seem to be creating a much more 'map savvy' generation of online users – I think this could be really beneficial because they're educating our users about mapping technologies and interfaces as well as making it possible for ordinary people to create geo-referenced content.

Flickr have also introduced stats for Pro accounts, which will make evaluating the use of our content a lot easier.