#CitizenHums : My thoughts on the 2 day event

From the periphery, I have always been interested in crowdsourcing and have often thought that it would be an interesting way to extend the www.bombsight.org apps that I developed back in 2012,.ie would it be useful for the public to engage in the crowd collection, co-curation of memories, photographs and documents to add to the formal archival resources. Furthermore, here at the CVCE we have been considering if it would be suitable to explore whether crowd-sourcing would be a helpful way of transcribing hand written and other documents,  from our Pierre Werner publications or even for error detection and validation of automatic OCR and entity extraction. In our histoGraph tool, the notion of expert crowds has been implemented to annotate and validate automatic face recognition of European Figures from face-recognition algorithms that processed our picture collection.

Therefore,  I was interested to find out more about crowd-sourcing projects and research in the Humanities. At the beginning of September I was lucky return to London to attend the Crowdsourcing for the Humanities in the 21st Century event, organised by Kings College London.

The theme of the 2 days of presentation and discussions was “Citizen Humanities Comes of Age: Crowdsourcing for the Humanities in the 21st Century”. The event was made up of a mix of interdisciplinary researchers from diverse backgrounds such as Archaeology, Ecology, Archival Research, Social Sciences and Humanities, which made for fascinating discussion.

For me some of the points of interest collated from discussions and speakers include:

  • There is a need for more research around the ethical considerations of crowd-sourcing activities. Is it possible to recognise the skills acquired and time contributed to such humanities projects?
  • Crowds alone do not sustain projects, it is essential to build a community from the bottom up (this reminds me of the very successful Open Street Map community);
  • Projects should not just be about technology and data capture – there needs to be greater emphasis on the research questions and managing social experiences.
  • There is an increasing need to nuture and prioritise human factors for improving likelihood of success.
  • Is it possible to develop crowd-sourcing pipelines that mix automation, human-intervention and machine learning to further develop crowd-sourcing activities.
  • Project design should enable the framing of dynamic research questions that emerge as projects evolve.
  • All projects have difficulty in overcoming the long-tail where the highest volume of contributions is made by a small proportion of all the contributors.
  • Successful projects should consider integrating a task ecosystem that provide a mix of different tasks taking different lengths of time to complete- appealing to non-experts and experts depending of the type of task and so eliciting different types of contributions.
  • Lots of projects fail but rarely are there discussions of why or how they fail (for me I think lessons learned from failure can be as interesting as those that succeed)

crowsourcing pics

Making histoGraph open source

As many of you already know, histoGraph is a web platform designed to help researchers to explore large multimedia archives akin to what we have here at the CVCE. The histoGraph tool has two main goals:

  1. To enable users to find and identify the most relevant documents for research
  2. To discover  document connections between persons.

In the last few months our designer/developer Daniele Guido, has started the redesign and development of the tool: this screencast shows some of the new features of Histograph as well as identifying the benefits and limitations of the new design.

We can now explore the “neighborhood” of a person in terms of other co-occurring people and documents, follow the paths that connect one person to one another; or simply search for resources and pose questions related to the person or the document.

Still under development, the open source version of Histograph starts from this assumption and basically serves as recommendation system. The original database has been enriched with CVCE text documents and been transposed to a graph database in order to speed up all network related computation. Moreover, a simple and powerful text extraction chain has been added:  thanks to the powerful Yago disambiguation engine, all text captions, titles and abstracts have been annotated with Named Entities in at least two languages.

Additionally, dates of both pictures and text documents have been reconciled to ISO standard; geographical places to latitude and longitude by using Geonames and Google Geocoding API; finally, person entities have been enriched with related dbpedia information. We will shortly be updating the data curation workflow to  assess and validate the quality of the automatic extraction of these entities and we plan further develop our concept of expert crowdsourcing techniques.

We will use the blog to update you on progress so watch this space.

DH Lab at DH Benelux

On 8–9 June 2015, the Digital Humanities Lab at the CVCE was very happy to participate in the second DH Benelux Conference hosted by the University of Antwerp.  See the presentations below. We contributed a total of four paper presentations and one poster describing our work. Dr Catherine (Kate) Jones discussed the concept of developing tools for personal narrative making by reusing CVCE cultural heritage objects, whilst Dr Marten During and Dr Florentina Armaselu presented the first outcomes of exploring TEI annotation and network analysis within diplomatic documents. Dr Lars Wieneke on behalf of Dr Florentina Armaselu described the results of our recent collaboration with the University of Pisa and their EVT (Edition Visualisation Technology) software for the creation of new TEI encoded digital editions.

MyPublications: Enabling personal authoring and narrative making


Europe’s Beginnings through the Looking Glass: Publishing Historical Documents on the Web Using EVT


Kate also presented results from the project she led prior to joining CVCE, which developed an augmented reality application to explore historical map data from the WW2 bomb census in London. She also participated in the lively closing panel discussion on the role of digital technology within the broader humanities discipline. Finally, the Director of the CVCE, Marianne Backes, co-authored a paper exploring the role of DARIAH and the Benelux.


The conference was very well attended, with more than 150 participants, and represented the strength of digital humanities within the Benelux region and beyond. The conference showcased the depth and variety of digital humanities research through presentations in a number of related themes: (1) Cyber Culture, (2) Reflection & Criticism (3) Distant Readings, (4) Linked Open Data, (5) Digital Scholarly Editing, (6) Curation & Collection, (7) Geospatial Applications and (8) Networks & Topics. Excellent keynotes were delivered by William Noel, Director of the Kislak Center for Special Collections, Rare Books and Manuscripts at the University of Pennsylvania, and Elena Pierazzo, Professor of Italian Studies and Digital Humanities at Stendhal–Grenoble 3 University.

Next year the CVCE is looking forward to co-hosting the conference with the University of Luxembourg.

Developing personalisation tools for ePublications

Digital Tools_51_43_42The CVCE has been working on the development of digital tools that enable our users to personalise content from our virtual research infrastructure on CVCE.eu. We are developing the idea of a Digital Toolbox where users will be able to customise content from all CVCE ePublications. The mid-term goal of the Digital Toolbox is to provide a personalised research infrastructure where users can create, collaborate and comment on their own personal ePublications.

Structure of an ePublication using MyPublications tool

Structure of a personal ePublication using MyPublications tool

In the later part of 2014 we launched, in beta version, the first version of the Digital Toolbox.  We have concentrated efforts on the development of the MyPublications authoring tool, development using a recently implemented Scrum methodology. The MyPublications tool will enable users of CVCE.eu to write ePublications on topics of interest based on personalised research, teaching and learning needs by organising and structuring CVCE resources, such as historical documents, press articles, photographs and other multimedia material alongside your own text. The tool includes a presentation viewer, a sim ple interface where you can sequentially step through your ePublication (like a book!).  In the coming weeks the developer of the tool, Frederic Reis, will also add more features including a share link feature which will enable a MyPublications to be shared with colleagues and friends.

To find out more have a look at our first introductory video.

What about the technology underpinning the tool?

simplified model of mypublications tech infrastructure

Simplified model of MyPublications technical infrastructure

In diagram shows a simplified model of the MyPublications technology derived from the research infrastructure. The CVCE data repository uses Alfresco content management system to manage, store and retrieve a wide variety of cultural heritage objects such as documents, photographs, cartoons and videos.  These objects are then served to the Java based open source Liferay documents and media portal which delivers our CVCE ePublications to the web and our half a million website users.

The tool then works through the client side and is written using a combination of HTML 5, CSS & JavaScript to create an interface that calls saves and embeds links to any type of cultural heritage object found within the CVCE collection. The resulting MyPublication is essentially a series of sequential object links and original texts (sections, sub sections and paragraphs) all stored alongside the metadata in a MYSQL database.

We will launch the full, tested version of the MyPublications tool in the early part of 2015. In the meantime if you want to be a beta tester please feel free to sign up.

Scrum Methodology & it’s Practical Use at CVCE

Services and tools developed by the Digital Humanities Lab for CVCE.eu must cater to a variety of target audiences and their specific needs. More often than not, at the start of projects, these needs are not yet fully developed and are highly uncertain. So whilst we have a detailed overview of a product and a corresponding set of basic user requirements, it is not always possible to plan at the the inception of a project exactly what detailed form the end-product will take.  The user needs and knowledge become clearer and more established during the iterative process of technological development, which can stimulate creativity, discussion and facilitate the emergence of further user requirements. Therefore, for future projects starting between 2014 and 2017 we are looking to adopt a pragmatic methodology that enable us to efficiently move from an idea to a stable product.

We aim to use an empirical approach for developing projects and their associated products, through the adoption of the Scrum framework. The Scrum framework is a methodology for developing and sustaining complex products. It is suited to circumstances where solutions are required for complex adaptive problems requiring creative approaches (source? Mountain goat?). It is based on the following realisims:

  • You can’t plan your way out of uncertainty
  • The point in a project when you know the most is when it is complete

With a focus on highly user-centred development process that places user needs (and their uncertainty) at its heart, scrum affords transparency of the development cycle to our internal stakeholders (researchers in European integration studies) and will ensure best value to our external users (scientific communities, teaching and lifelong learning communities, society at large). It will allow inspection of products regularly, starting at an early development stage.  It also provides a framework to adapt products based on the users’ knowledge gained during the development process. Thus, it should help us deliver more robust, useful and usable solutions.

What is Scrum?

In essence Scrum is a methodology based on AGILE, where product requirements and solutions evolve as a consequence of collaborations between teams, product owners and their stakeholders. Scrum can be applied to many different types of projects but is commonly adopted by software development projects as it is highly suited for rapidly changing or emergent requirements, with the aim to create software delivers more value (Scrum.org) and can be summarised in the following high level, simple steps:

  • A Product Owner creates a prioritised, ordered list of items (akin to a wish list) known as a product backlog
    • Often the items are written as user stories
    • Items at the top of the list are written with more detail
    • The backlog is continuously evolving
  • Then in a sprint planning meeting, a subset of the items on the product backlog are selected  by the scrum team and placed into a sprint backlog.
  • The sprint backlog forms the tasks and activities of the a sprint
    • a 1 to 4 week period within which the items on the sprint backlog must be completed
  • During the 1-4 week sprint period a 15 minute daily fixed scrum meeting
    • Same time, same people, same place .
    • The aim is to understand:
      1. What work was completed yesterday?
      2. What is planned for today,
      3. Are you facing any problems?
  • The ScrumMaster helps resolve any impediments to progress during the sprint
  • Following the completion of a Sprint, there should be a potentially releasable iteration of product that can be shown to the product owner and stakeholders during a sprint review
    • If the sprint is two weeks long then it is suggested a sprint review meeting should last two hours.
  • A sprint retrospective is held to review the sprint
    • what went well?
    • What could have been improved?
    • What actions to take in the next sprint.
  • Repeat process for the next set of items ordered at the top of the backlog

    Scrum Methodology , source: click on image


How will we implement Scrum methodology at CVCE?

The first step is to identify a few projects which would really benefit from a Scrum approach and begin from here, then we will use a snowball effect to then adopt it for other projects after the benefits and business value are clearly visible to all our stakeholders. I have been on the Scrum Product Owner training course (which I enjoyed immensely) and my college has participated in the ScrumMaster training course.

At the CVCE there will be three main departments involved developing projects and products using Scrum

  1. European Integration  Studies (EIS)  – comprising of social science and humanities researchers
  2. Digital Humanities Lab (DH Lab) – comprising of digital humanist researchers who are the interface between EIS and DevOps.
  3. Development & Operations (DevOps) – comprising of computer scientists and media technologists

The project that we have selected to be our pilot for the adoption of Scrum is one where we are working on evolving and improving the interface, our data model and data enrichment process for the CVCE document object repository. As part of this evolution we are integrating a metadata zone within the interface and data model to enable EIS to easily provide metadata in line with the international standards of Dublin Core, for all the objects they create. This will be combined with a citation generation service to provide internal users (CVCE historians) of the repository and readers accessing the repository via the website with automatically generated citations for all resources, conforming to ISO 690 numeric French standards. To complement this, there is a need to develop a metadata quality control process to ensure consistency of the authority lists that are created as part of our metadata policy.  It is this project that we are implementing with Scrum.

citation user stories

Example of our organised user story descriptors for items associated with citations

A member of the DH Lab is the product owner ,EIS our stakeholders and various staff from DevOps and the DH Lab comprise the scrum team. The product backlog is emerging based on sets of user stories derived from interviews with users and observations of how they are currently working and the strategic objectives. We are working in 2 week sprints cycles and are currently half way through sprint 4.

To mange the scrum we are using JIRA, to both manage the backlog and organise and plan our sprints.

Thus far we are enjoying working in this way and our stakeholders appear happy. I will post further updates and details of our progress as they emerge.

Talk of Europe – Travelling CLARIN Campus

In October we will be participating in a one week creative camp, organised by the Talk of Europe – Travelling CLARIN Campus project (ToE-TCC). Funded by Dutch agencies NWO and OCW, the project follows up on the PoliMedia project, which linked entities in Dutch parliamentary records to corresponding media sources archived by the Netherlands Institute for Sound and Vision.

The CVCE was successful with two proposals submitted to ToE-TC, both focused on exploring social graphs.

  1. Visualising Co-occurrence networks of the plenary debates of the European Parliament
  2. Exploring the potential of multimodal graphs for modelling European Plenary Debates

We are excited to participate in travelling campus, as we are looking forward to finding new ways of structuring parliamentary data. We look forward to exploring how we can use such tools, techniques and data structures to shape future developments of the CVCE research infrastructure and develop new ways of exploring our collections.

There are three aims for the one week event, scheduled for 6-10 October 2014:

First Aim: “To translate the proceedings of the European Parliament debates (europarl) to CLARIN standards. In other words, the europarl data is curated to linked data so that it can be linked to and reused by other datasets and services. An interesting aspect of this dataset is that its available in 21 languages, enabling to link to other datasets in as many different languages. (…)

Second Aim: To research how this linked data can be exploited for use by humanities scholars. The combination of linked datasets with the base available in 21 languages should allow research questions not yet feasible. As such, interlingual comparative research through digital tools becomes possible. In comparison, where PoliMedia focused on how Dutch debates were covered in the Dutch media, the europarl data could be analysed to compare how European Parliament debates are covered in Dutch versus Polish media. Political scientists could analyse how subjects from EP debates relate to subjects in debates of national parliaments. In order to build tools helpful, we will investigate user requirements and possible research questions from humanities scholars. This second aim will be undertaken by Max Kemman, Martijn Kleppe and Henri Beunders from the History department of EUR.

Third Aim: To spark a transnational, European collaboration to create tools for scholars to analyse linked datasets. To this end, three meetings of a week will be organised in which teams from all CLARIN countries are invited to participate. Think hackathon, but a week long and with a European focus.”

The European parliamentary debates have been processed and are now available in RDF format. RDF, commonly associated with the buzz-word “Linked data“, “is based upon the idea of making statements about resources (in particular web resources) in the form of subject–predicate–object expressions. These expressions are known as triples in RDF terminology” (Wikipedia article on RDF). RDF is highly flexible and allows users to formulate very precise queries on their datasets.

A simple social network graph

The first proposal, in cooperation with Dr Fintan McGee of our neighbouring research centre CRP Lippmann, will explore the extent to which multimodal network analysis can be fruitfully applied to the study of the parliamentary debates. Where one-mode network visualisations can only represent one type of actor in a network (e.g. people), multimodal networks are capable of representing multiple types of actors, e.g. people, institutions, texts and places. This approach bears the potential to represent more closely the complexities of real-world networks.

An example of a geographic social network
(source: Nodegoat).

The second proposal was submitted with Lab1100, a Dutch company which develops Nodegoat. Nodegoat offers its users with simple yet powerful instruments to create relational databases online and visualise their data in form of maps and network graphs. Lab1100 will develop a way to link ToE-TCC data directly to Nodegoat using the formers SPARQL endpoint. This will ensure very quick and highly flexible data import and visualisations of the data.

CVCE researcher Marten Düring and Frédéric Allemand, Coordinator of the European Integration Studies Department at CVCE will participate in the project.

Digital Humanities LAB at CVCE – why we have a blog

The Digital Humanities lab at the CVCE is an interdisciplinary team, whose goal is to participate in the development of the CVCE’s innovative research infrastructure for European integration studies.

This blog will provide the DHLab with a forum for the exchange of information and ideas associated with our ongoing digital humanities projects. It will provide the opportunity to discuss the CVCE’s projects we are working on and the issues and challenges we face developing tools and techniques for multi-lingual digital  research environments using a wide variety of data types (pictures, text, video, maps cartoons etc) for developing novel enhanced publications.