ICTD 2009 Recap

This Post By: Jahanzeb Sherwani

Jahanzeb is a recent Carnegie Mellon graduate whose doctoral work dealt with speech interfaces for low-literate users in Pakistan. Here he presents a personal perspective on last month's ICTD conference.

The ICTD conference was held at Carnegie Mellon University's Doha campus in Qatar last month. It was the third such conference, after the first one at Berkeley, USA in 2006, and the second in Bangalore, India in 2007.

For me, the conference in many ways represented a sort of coming-of-age of the field of ICTD. It wasn't just that there were many more papers submitted than ever before -- and there definitely were -- but that the questions being asked had changed in many ways. Some of the papers asked questions we should have asked many years ago: a paper presented by Bill Thies (currently at Microsoft Research, India) compared the accuracy of data entry using forms, SMS and voice-based live operators, and showed that a call center could be much more preferable than alternative mechanisms. Similarly, Aditi Sharma's presentation comparing the use of touch-tone and speech for information access by HIV caregivers in Botswana (work done at the Meraka Institute in South Africa) showed how such systems could potentially be used in the real world, and the costs and benefits of both interfaces. Given that this is the same space that I've been working on for my PhD, it was great to get a different perspective on the same issues, and see how there is surprisingly high commonality in our contexts, even though our sites are in different continents. Rowena Luk's presentation on a comparison of two case studies in ICTD-for-health again showed how the field had advanced; while such case studies were regular attractions at ICTD, a comparison showing similarities and differences was clearly a first, and I hope that there are many more such studies to come. My favorite paper though was by the IBM Research India team, presented by Arun Kumar, on a phone based content generation and dissemination system for and by rural end users. This was the first time that I saw the classic inverse-hockey-stick graph of usage over time in an ICTD project. Everyone has been talking about the importance of local content (and I've seen the need for it in my work too) for years now, but this work was the first of its kind that designed, developed, and tested a system that actually enables users to create and share their content themselves. The implications of this work are immense, and I really hope that others are as excited by this as I am.

Beyond the papers though, what really made the conference worthwhile was the inter-personal exchange. Perhaps more important than the crisp, academic results presented in the main presentation hall, are the micro-exchanges that go on between researchers more informally. I was really excited to learn that Ilda Ladeira, a PhD student in Computer Science at the University of Cape Town, South Africa and an intern at Microsoft Research, India, was working on experiments to tease apart the effect of dramatic narrative in information access by low literate people. So it isn't a question of getting information to the end user, but the context in which you couch that information, and the dramatic embellishment around the information. To me, that's really revolutionary, as it's moving beyond the simple ideas we started out with in the field, and turning the concept of information access on its head in a way that could have profound implications for both the developing and developed world. This builds upon previous work done by Indrani Medhi and others at the Technology for Emerging Markets research group at MSR, India, and was being done under the supervision of Ed Cutrell, an HCI researcher at the VIBE group at Microsoft Research, Redmond, USA. Having people like Ed spending time working on HCID issues is also really great to see, and I think that this cross fertilization of ideas and experiences between researchers straddling the boundaries of Western-facing and emerging-market-facing projects is really going to benefit the field in a big way. It was also great to sit in on a conversation between Revi Sterling and Thomas Smyth (both working on social & civic media and getting communities talking) -- even single phrases from this conversation gave me what I needed to start thinking about a number of issues I'd never considered before (e.g., the phrase "public sphere"). And of course, ICTD is always a great venue for reconnecting with colleagues, to catch up on what they're doing, and to get feedback on what you're doing: Divya Ramachandran (CS PhD student at UC Berkeley), Brian DeRenzi (CS PhD student at University of Washington) and I met up several times to discuss our work on cell phones for community health workers, and came up with a number of exciting ideas for subsequent work. Even these exchanges seemed more energized this time around than ever before. However, such interactions are limited to those you know, or a friend-of-a-friend -- it's difficult to meet people outside one's immediate social network. One suggestion I have for the next conference is to publicize the participant list to all participants, including a short (1-2 sentence) statement by each participant on who they are and who they're interested in meeting -- there is so much potential synergy at ICTD, and encouraging introductions and cross collaboration could make a big difference.

This was also the first ICTD that hosted workshops. Being a co-organizer of one of the workshops (the Young Researcher's ICTD workshop), I was glad to see that there was healthy exchange between students and faculty on issues that are often not given the space to be publicly discussed: for instance, Yaw Anokwa (Computer Science PhD student from University of Washington) talked about the tension between doing work that would normally be considered worthy of a CS PhD, versus doing work that is relevant and important for ICTD, and Tim Unwin (Professor at Royal Holloway, University of London) commented that it's hard to imagine such tensions still exist in the CS academic community. What was interesting was not that two members of the same academic community had two such opposite views, but that these issues were discussed for the first time at such a venue, and I truly believe that it is only through more such exchange that such issues can eventually be resolved. One of the goals of our workshop was to encourage inter-personal exchange, and Divya Ramachandran (co-organizer of the workshop) had designed a number of activities including speed-dating and story-sharing that got both groups and individuals talking to one another. Based on the survey feedback we received, attendants thought the workshop was really useful -- and I hope that workshops become a permanent part of ICTD, as they provide a crucial counterpoint to the formal discussions in the presentations.

All in all, I left ICTD with a renewed sense of hope, that we as a community were getting somewhere, and that amazing people are doing amazing things around the world. Kudos to Rahul Tongia, Richard Heeks, Bernardine Dias, Sarah Belousov, and everyone else on the organizational team for making the conference a resounding success. Let's see if Seattle or London can take the conference even further. I look forward to seeing you there.

Notes on Participatory Evaluation and Sustainability

This Post By: Ann Light

Ann is a reader in interaction media and communication at Sheffield Hallam University in the UK. She brings us this post about the merits of participatory approaches to evaluation.

At the HCI4D workshop at CHI 2009, I was struck by the similarity between the questions we were asking about evaluation of work using ICT for development (as in: what makes a successful project?) and a report I recently co-wrote on user-centred design and social justice. Since international development and social justice goals are not far apart, maybe I have no reason to be surprised. But I thought that the links and references we provided for the report might be useful to share here. So the following is a couple of sections from the Futurelab report by Ann Light and Rose Luckin, Designing for social justice: people, technology, learning, to be found in its entirety at: http://www.futurelab.org.uk/resources/publications-reports-articles/opening-education-reports/Opening-Education-Report1128

Here goes:

Participatory evaluation

UNESCO has backed the benefits of ethnographic action research with its reflective and continuous approach to project evaluation. Tacchi et al (2003) reject an approach that measures a ‘baseline’ and then subsequently monitors and evaluates changes from that baseline to measure the effectiveness and impact of a project. They advocate a “research culture through which knowledge and reflection are constantly fed back by all parties in ways that help the project to develop” (2003: 3-4). Involving all participants in assessing the development of the project fosters ownership, increasing accountability and commitment of all involved. Distributed ownership in and of itself improves the chances for a more longitudinal approach to evaluation, overcoming the short-termism of much evaluation and assessment work. This reflects the likelihood for longer-term impact as ownership also increases the chances of sustainability within the locale or community that has taken it up.

Another voice promoting democratic evaluation methods is Hall's (2003) who has named her approach ‘process-generated evaluation’. This stresses the involvement of all stakeholders and allows for a negotiated form of evaluation that emerges from the activities of the group. In allowing for greater engagement, it complements participatory design well and its emphasis on learning through activity rather than fixed goals and targets offers the flexibility needed to be responsive to the forming and reforming that accompanies listening to all participants.

Themessl-Huber and Grutsch (2003) note that in addition to supporting decision-making and problem-solving, evaluation can have an empowering effect for projects’ users. They show how the changes in stakeholder behaviour and the changing roles of evaluators can influence the evaluative process, arguing that it is an illusion for evaluators to think that they are in control throughout the entire evaluation, any more than those who commission the project will be. They suggest that these shifts are themselves important outcomes of the act of evaluating and deserve the same attention as other effects (2003: 108-109).

As mentioned, Holden (2004) makes the point that ‘regeneration’ is not enough without prosperity and increased community wellbeing. In choosing evaluation criteria, it would be possible to focus on the achievement of processes and products that are the means for change, instead of the end goal. This introduces a danger of overlooking the benefits actually achieved or focusing on short-term outcomes at the expense of valuable progress. By contrast, the methods described above have measures within them that reduce the dangers of evaluating the conceived means of making change instead of the change itself. First, the likelihood is that open and participative evaluation activities act to embed the changes being discussed through the process of reflection that evaluation should entail. Second, an unimpressed user-designer community often reveals itself in open evaluation processes: using blunt questioning of purpose and value to raise concerns. The very act of joining in a participative evaluation process is a marker of commitment and absence of comment may be an implicit comment on the success of engaging people. Thus, flexibility and openness are rewarded as qualities of a project, but only when they too serve its higher goals. There is little room here to look at the effect of overly-restrictive reporting expectations from funders, or funding conditions that curtail collaborative exploration of effective change mechanisms. It is enough to note that both long-term engagement and evaluation and the accompanying pronounced shifts in the nature of the work (so that actual on-the-ground desires are being followed) are made difficult if support is not sensitive to the value these can supply.

Sustainability

Sustainability is an issue for any social justice project at some level. Whether sustainability is about taking the next step, maintaining the achievements of the project, or ensuring that the next generations can also benefit, there is a challenge to giving life to the next iteration once the funding has run out or the people who championed it have moved on. Gaved and Anderson (2006) note that people bringing initiatives into a community “need to have an exit strategy and be aware of the effect they may have if they leave”, whereas “[g]rassroots initiatives may be more sustainable, as they are supported from within the community usually on a financially self-sustaining model, but need to ensure a sustainable social model as they draw heavily on the community’s social capital, which can be spent as well as accrued through the maintenance of the initiative” (2006: 27-28).

The Department for International Development (DFID) provides guidance on sustainable ICT projects for development (www.sustainableicts.org) and offers ideas that could apply to any social justice project. Amongst the case studies it describes is CDI, the Committee for Democracy in Information Technology. CDI facilitates the running of Information Technology and Citizens Rights schools (ITCRSs), providing the equipment (hardware and software), training of local educators, and local administrative and technical support. The schools are self-managed and self-sustainable, as are the regional CDI offices that support and monitor them (www.sustainableicts.org/infodev/CDI.pdf). Developments like this one, that run so much on local enthusiasm, bring with them the question of how you maintain that degree of commitment. “Unsustainable systems deplete or run down capital, spending assets as if they were income”, say Batchelor and Norrish (2002). For CDI, a key factor in avoiding this has been the link back into the needs of the communities served: “CDI trains the future educators of the schools who, in turn, will train others back in their communities. The fact that the local people are heavily involved in the process means that the courses offered are tailor made and relevant to the community.”

Batchelor and Norrish (2002) define the many forms of capital that they see as needed for sustainability, including financial, physical, social (structures and institutions which support uptake) and human (actual people). Arguing that to become embedded, information must become knowledge held by local people, they suggest that “[t]he critical question in all this is where is information to come from if it is to be relevant and usable to local populations and where is the support to come from if information is to become knowledge? Are information only projects sustainable? Or is the step to knowledge vital for sustainability?”

Similar issues arise when scaling up projects from a pilot to a programme with wider reach. Clarifying which aspects of a project are scalable is crucial to meaningful transformation. Participatory design is hard to scale to large generic projects, as we noted above. So, in evaluating goals, there is little difference between planning for sustainability and planning for scaling. Key values need extracting that can be protected. The rest is merely the conduit for change and can be adapted pragmatically for effectiveness.

There is evidence that the model of participatory design used affects the sustainability of a project. Carroll et al (2000) compare their long-term process in Virginia Tech, which ran over five years, with more common short-term ‘duck in-and-out’ techniques. “Participatory design is fundamentally a process of mutual learning, and thus of personal development for participants. But it is often exemplified by rather singular and ephemeral learning interactions” (2000: 249). The Virginia Tech researchers engaged in long-term cooperative design interaction with roles and responsibilities for participants that developed as their interest and experience developed.

References

• Tacchi, J, Slater, D and Hearn, G (2003). A User's Handbook Developed to Innovate and Research ICT Applications for Poverty Eradication UNESCO (eprints.qut.edu.au/archive/00004399/01/4399.pdf)
• Themessl-Huber, MT and Grutsch MA (2003). The shifting locus of control in participatory evaluations. Evaluation, 1 (9), 92-111
• Hall, R (2003). Developing forms of democratic evaluation. Interrupt Symposium, Artist as Educator, UCE, Birmingham, 12 March 2003
• Holden, J (2004). Capturing Cultural Value. London: Demos (www.demos.co.uk/publications//culturalvalue)
• Gaved, M and Anderson, B (2006). The impact of local ICT initiatives on social capital and quality of life. Chimera Working Paper 2006-6, Colchester: University of Essex
• Batchelor, S and Norrish, P (2002). Sustainable Information and Communication Technology (ICT): Thoughts by Batchelor and Norrish, April 2002 (www.sustainableicts.org/Sustainable.htm)
• Carroll, JM, Chin, G, Rosson, MB and Neale, DC (2000). The development of cooperation: Five years of participatory design in the virtual school. In Proceedings of DIS. New York: ACM Press, 239-251

The Two-Month Lesson

This Post By: Brian DeRenzi

Brian is a computer science Ph.D student at the University of Washington, currently living and working in Tanzania.

I am in Tanzania now working on designing and developing a mobile-phone based support tool for community health workers. I've been in Tanzania since the end of October and will be here likely until next November some time. The work has been going well and we have phones deployed with multiple organizations. However, I'm not going to talk too much about the application or design process today.

Instead, I want to talk about a phenomenon we've noticed that pops up after 1.5-2 months of use in the field. (Credit goes to my colleague, Gayo Mhila, for recognizing the generality.) We like to call this the "I-know-how-to-use-the-phone-and-can-break-it-now" phenomenon, or "ownership phenomenon" for short. The basic process is something like this:

- Phones are distributed to some field workers (in our case Community Health Workers, or CHW for short)

- There's a small (one week?) initial training hump while the users try things out in the field and adjust to using a "fancy" phone ($100-120 in our case)

- Things run smoothly (hurray!) for 4-8 weeks

- The "ownership phenomenon" hits!

We start to see phones coming back with custom backgrounds, new themes, pictures being taken (mostly of themselves or their children), ring tones being acquired, videos being recorded etc... In one extreme case, we found that the CHW had actually managed to delete the application, then somehow created a nested set of folders (e.g. Applications -> Amy -> Amy -> Amy -> Amy -> Amy) while she tried to log back in (name was changed).

The point is that there are a host of issues that pop up only once the system has been running for a while. This is important because I think it's not uncommon for a researcher to see things going well for a month, collect their data, and be happy with the result. As researchers, we need to make sure that our systems have time to reach steady state before we start to draw any conclusions. I'm just rehashing what we've all said before, but as always, what happens in the field is different from what happens in the laboratory.

CHI 2009 Recap

This post by: David Hutchful

In this post, David Hutchful of Microsoft Research India brings us a recap of this year's CHI conference from an HCI4D perspective, including a review of the two-day HCI4D workshop at the start of the conference.

The Workshop

About 37 people gathered to attend the HCCID workshop in Boston. The workshop was a good opportunity to meet and discuss ICTD related issues with others in the CHI community interested in ICT4D.

After self introductions and position paper presentations on the first day, participants engaged in various small and large group discussions, one of which was a panel discussion on the motivations, strengths and weaknesses of the usual stakeholders of ICTD projects (see appendix A for discussion notes).

John Thomas opened the panel with a narration of the story “Who speaks for wolf?”, a reminder to always consider all potential stakeholders (accordingly, an empty chair was added to the panel table to represent any unrepresented stakeholder ). The general conclusion, given the constraints of the stakeholders, supported a continuance of the inter-disciplinary approach to ICTD with the goal of improving collaboration within the groups and aiming for sustainability in projects.

The second day begun with a rousing round of speed dating: an opportunity to sit and converse with individuals with whom one had no previous affiliations. Following this were group discussions on various themes. For the first set of themes (i.e. methodology, evaluation impact, holistic design, eco-system, end users and commercial view of development) the groups came up with one or two “big questions” and possible answers. The second set considered questions on external outreach and internal collaboration efforts in the community in addition to the future trajectory of the community (see attached file for discussion points).

The day ended with open discussions on various HCI/ICTD issues including how best to use the resources of SIGCHI to promote the agenda of the community (e.g. involving economic/social/cultural development domain experts in our conferences; encouraging local HCCID meet-ups ala CHI).

One interesting flash point focused on the role of the end user in ICTD conference activities. Some were of the view that it would be beneficial to have end users involved at conferences and workshops, while others argued that although this sounded like a good idea, it is in fact a very “romantic” (it very much appeals to the community’s drive to be human centered and holistic in its approach) and not practical one. The latter group argued that the benefits would accrue primarily to the community and not necessarily to the end-user. In addition, they argued that events like conferences and workshops are opportunities to receive critical feedback from peers and it is not immediately clear whether the end-users will be able to serve within this capacity. There was not enough time to come to a definite conclusion.

Overall, new comers found the workshop a quick introduction to the people, organizations and issues involved with ICTD. For veterans, this was more of a refresher course (euphemistically speaking☺) and a good opportunity to renew/solidify old ties and make new ones. For all, it was an excellent opportunity to grapple with the important issues of ICTD and to learn from the collective experiences of the participants.

The Conference

Judy Olson led off the main conference with a plenary speech on what she calls “Social Ergonomics.” Building on the theme “even small distance matters,” she described how the structure of physical space affects peoples’ behavior and how people are not aware of these effects until they are violated. For example, we all have a personal space demarcation into which we only allow intimate friends: we only become aware of this space when someone not so close to us invades it. Judy then talked about the differences in behavioral responses due to cultural influences and lastly, how we can better design and situate technology in our physical space to accommodate peoples’ natural behavior in space.

The rest of the conference was choked full of poster sessions, panels, courses and paper presentations. Matthew Kam, Neil Patel, Indrani Medhi, Leah Findlater, David Frohlich, Neema Moraveji and Thomas Smyth’s works (sorry if I missed any others) were some of the ICTD related papers presented. Given below are the titles of the papers along with the names of the presenters:

• (Matthew Kam) DESIGNING DIGITAL GAMES FOR RURAL CHILDREN: A STUDY OF TRADITIONAL VILLAGE GAMES IN INDIA
• (Neil Patel) A COMPARATIVE STUDY OF SPEECH AND DIALED INPUT VOICE INTERFACES IN RURAL INDIA
• (Indrani Medhi) A COMPARISON OF MOBILE MONEY-TRANSFER UIS FOR NON-LITERATE AND SEMI-LITERATE USERS
• (Leah Findlater) COMPARING SEMILITERATE AND ILLITERATE USERS' ABILITY TO TRANSITION FROM AUDIO+TEXT TO TEXT-ONLY INTERACTION
• (David Frohlich) STORYBANK: MOBILE DIGITAL STORYTELLING IN A DEVELOPMENT CONTEXT
• (Neema Moraveji) A MISCHIEF OF MICE: EXAMINING CHILDREN’S PERFORMANCE IN SINGLE DISPLAY GROUPWARE SYSTEMS WITH 1 TO 32 MICE
• (Thomas Smyth) DESIGNING FOR AND WITH DIASPORA: A CASE STUDY OF WORK FOR THE TRUTH AND RECONCILIATION COMMISSION OF LIBERIA

Other highlights included a presentation by Jan Chipchase on Nokia’s “Observe then Design” method. He went into some detail about his in-field (he called it a “deep dive”) experiences: building a team, data collection, the ethical issues he encounters, getting most out a culture in 10-14 days and some of appropriation of technologies he comes across, which Jan calls “street hacks,” (e.g. a contraption that allows mobile phone SIMs to be hot-swapped).

The conference ended with a note by Kees Overbeeke, an industrial designer from Eindhoven University of Technology, who spoke on the integration of science, design and engineering as a means to innovation in the future. His talk was based on his experiences on carrying out such integration in the Eindhoven industrial design program.

Over all the conference was well attended: about 2000 attendees from over 40 countries (the representatives from the two African countries present were attending the HCCID workshop). This was again another good opportunity to touch base with ICTD folks and catch up with all the exciting projects the CHI community is undertaking.

Welcome to the HCI4D Coblog

This Post By: Thomas Smyth

HCI4D is a young, diverse, and multidisciplinary field of research which explores the design of digital technologies for international development. The idea for this community blog (coblog) was conceived at a recent meeting of HCI4D researchers as a way for people working in the area to make connections, stay in touch, and keep abreast of the latest news and discussions from a range of perspectives.

The blog will work as follows. Anyone doing research related in some way to technology design and international development is encouraged to contribute when they can. Posts needn’t be long or profound, just timely and interesting. Topics might include but are not limited to:

• reports on ongoing research
  
• experiences from the field
  
• reviews of relevant books or articles
  
• recaps of events such as workshops or conferences
  
• opinions on relevant issues of the day
  

... or anything else that is relevant to HCI4D and serves to stimulate and connect.

The blog will also have comments enabled, so that discussions on interesting posts can ensue.

As moderator of the coblog, I have assumed the responsibility of soliciting contributions to it. I will do my best to get things moving and stimulate interest, but ultimately, this blog is what the HCI4D community makes of it. That said, I am confident that it will grow into a vibrant forum for discussion and exchange.

Stay tuned in the coming days for recaps of the CHI and ICTD conferences, as well as a report from Brian DeRenzi in Tanzania.

Please email thomas.smyth@gatech.edu if you would like to contribute.

Sincerely,
Thomas