Unit 3
How David Eaves teaches Unit 3 (part 2)
Learning: Lean Startup
What is this page?
This is a detailed breakdown of how David Eaves, a Lecturer at the University College London's Institute for Innovation and Public Purpose (UCL IIPP), teaches the contents of Unit 3 of the open access syllabus developed by Teaching Public Service in the Digital Age. Read how part one of Unit 3 is taught here.
This page is part of a series of twenty-five classes that David developed originally for the Harvard Kennedy School's master and executive education programs, where he taught for eight years, and are now taught at UCL's master and applied learning programs.
We believe presenting diverse ways to teach the syllabus will help others adopt and teach the material in various contexts. See here how Konstanz University's Prof Ines Mergel teaches the same unit.
Who is this page for?
This page was developed for university faculty who teach public administrators or master's levels students in public policy and public administration. This material may also be suitable for teaching to upper year undergraduates.
Class Overview
In the previous class we explored the difference between project management methodologies that favor planning (referred to in this course as waterfall-like approaches) and methodologies that favor learning (referred to in this course as agile-like approaches). As discussed, governments are predisposed to adopt waterfall-like approaches for much of their work. This is because, under the right circumstances, waterfall-like approaches are both efficient and align with government's practices around budgeting and accountability.
However, building services and processes in a digital era tend to not conform with the preconditions that favor waterfall-like approaches. Agile-like approaches, as seen in the previous class, can feel slower, but are designed to help embrace uncertainty by testing ideas quickly with citizens to see if the service or policy will work. This need for increased learning and iteration is as true in the private sector as the public sector and is why agile-like approaches have been ascendent. Given that most government services will be digital - or have significant digital components - this has significant implications.
In this class, David seeks to explain one agile-like approach - the Lean Startup methodology - and explores what it means to implement it from a managers perspective. Building on the previous class the goal is to have students imagine managing a process that develops a hypothesis about a service or policy, develops a test of that hypothesis and gathers data to then improve or rethink the hypothesis. As David notes, this is often a more time consuming and resource intensive approach than waterfall, but it is a more efficient use of time and public resources as it both **increases the odds of a success and helps identify failures earlier on. He also discusses the challenges governments face in the implementation of these complementary approaches.
This Class' Learning Objectives
By the end of this lecture students should be able to:
Explain what iterative approaches to project management are, and describe where they come from.
Describe some of the characteristics of 'Fake Agile' projects whereby traditionally managed projects and overall governance appropriate the language of iteration but not its practices or impact.
How this class relates to the Digital Era Competencies
💡 This class has a specific focus on Competency 4 - Iteration. See all eight Digital Era Competencies here.
Assigned Reading and Practical Resources
As they work through the readings in advance, students should have in mind the following questions to help them prepare for class:
Think back on past work - what was the fastest, cheapest way you ever tested an idea? What enabled you to move so quickly and cheaply?
Think of an IT project you have been involved in that failed: How might you have converted that project to take a more agile approach?
What hypothesis would you have developed?
Can you brainstorm a minimum viable product that would have enabled you to test this hypothesis?
Governments are uniquely positioned to quickly scale services. How does this ability support or undermine the governments ability to test hypotheses?
Core Reading (Required)
Hypothesis-Driven Entrepreneurship: The Lean Startup (2011), Case Study by Thomas Eisenmann, Eric Ries and Sarah Dillard for Harvard Business School
Value Proposition Design [pages 10-25] (1999), Handbook by Alex Osterwalder and Yves Pigneur for Strategyzer
Advanced Reading (Optional)
PDIA: A Journey of Discovery (2018), Video by Harvard's Building State Capability (BSC) Center ****
Detecting Agile BS (2018), Guide by US Department of Defense
The Lean Startup **(2011), Book by Eric Ries
Detailed Class Breakdown
Class plan: 75 minutes
See David's slides for this class.
The sections below describe the dynamics of each part of the class:
Segment 1 - Recap of agile vs waterfall methodologies – 10'
This section's goal is to recap agile-like and waterfall-like project management approaches and highlight when agile is implemented incorrectly.
Purpose of this segment
In much of the tech sector, agile-like methodologies have become the dominant project management methodology for projects. This is because these organizations accept that highly digital projects often have many unknowns that need to be uncovered through processes that favor learning over planning. Agile-like processes use rapid iterations (often called sprints) to enable teams to find and resolve unknown unknowns and resolve or address them.
However, even when teams intend to use agile approaches they can fall into waterfall like practices, for example mapping out a plan, as opposed to defining testable hypotheses. The purpose of this section is to highlight the need for managers to be vigilant in identifying and addressing this problem. This is also a good place to highlight Detecting Agile BS reading.
Video of David teaching this segment
Reflections
To start the section, David goes back to the 2nd student exercise from the previous class when planned a hypothetical trip from St Louis, Missouris to America's west coast in 1804. Although students adopt different approaches and claim they want to learn and adapt, they often continue to plan and do not engage in creating an iterative process that will allow them to learn. Here, the instructor should highlight how critical it is to define and test a hypothesis. The previous' classes exercise provides an engage opportunity to push students on how they might do this in the context of the exercise. Whatever suggestions they bring, encourage them to think about which test or sprint they could use.
During the discussion, it's important to deconstruct two misconceptions about agile: that it doesn't require planning and that it's faster. In fact, a serious use of this approach requires intensive planning on how to test hypothesis, iterate and plan the new tests. In the end, agile approaches probably take longer and require more effort. But if done well, the chances of success are higher.
Segment 2 - Combining Agile and Lean approaches – 20'
This section's goal is to introduce the main concepts of the book The Lean Startup and how to combine them with agile approaches for better outcomes in the public sector.
Purpose of this segment
Two of the biggest reasons why government IT projects fail are that teams execute plans based on untested hypotheses about the users, and that the projects aren't valuable for citizens. The first problem can be addressed through agile methodologies, that when correctly applied enable learning circles to find unknown unknowns and adapt to better plans, iterating until the end. The Lean Start Up builds on this by providing a methodology for teams to iterate and test if the product or services is of value to citizens and ultimately increase its value or impact. The purpose of this section is to present these methodologies and to discuss when to use them in a government context.
Diagram showing three concentric circles. The smallest, central circle says 'Waterfall - We have a detailed plan and we assume it will work'. The next largest circle says "Agile - plans often fail. Let's test with users and make sure it works." The final, largest circle says "Lean Startup: We don't want to build a great product that no one needs. We'll test on the market and make sure it sells". Image produced by Jehan Gonsal and published in this article.
Video of David teaching this segment
Discussion
The section starts with a discussion about agile and lean startup approaches. A key criteria for deploying either of these methods is that the situation being managed is complex and unpredictable. This includes:
complex social problems
large or dynamic number of stakeholders
new services or organizations
changing digital landscape
After this initial discussion, David discusses two tools that follow the Lean Startup methodology and can be used in governments:
Problem Driven Iterative Approach (PDIA), developed by the Building State Capability Program at the Harvard's Center for International Development. This methodology has been adopted by many governments, especially in the developed world since it allows public servants to build autonomy and not rely on outside solutions.
Value Proposition Canvas (VPC). Here, instructors can highlight the dangers of working with surveys to build a VPC. Some of the problems are that citizens might not be able to articulate or imagine what they want or need. A more insightful way of building a VPC might be to observe people's behavior and interactions with prototypes.
Reflections
Finally, instructors can highlight that agile-like and lean startup approaches to managing projects not only help test hypotheses for plans and the value proposition with citizens, but also build trust and get buy in between stakeholders and the team using these methods.
The agile-like processes explained so far are fundamentally focused on a teams' capacity to execute and meet a goal.
Segment 3 - Applied Example: KNET - 25'
The goal of this section is to showcase an example of the Lean Startup methodology in a context students are familiar with.
Purpose of this segment
⚠️ Pre-work: In addition to the readings and preparation questions, this next segment assumes students complete a value proposition map of KNET, the Harvard Kennedy School's intranet. For more information for how to create a similar assignment, please see the assignment note - instructors should choose a system students are familiar with.
The purpose of this segment is to practice using a structured tool designed to help students surface users needs to better understand them. This understanding then services as a spring board to analyze how a service is, or is not, serving these needs and opens up possibilities for how it can be improved.
Video of David teaching this segment
Exercise
To open this section David highlights and shares examples of the completed assignment where students were creative and effective in documenting users needs. Next, he splits students in groups for 8-10 minutes to discuss the following questions regarding KNET:
what tasks are users trying to complete, but the service makes difficult?
what hypothesis do you have about how it could be improved?
how you might cheaply operationalize such a test and measure it?
A circular pie diagram with three slices. One slice is things the students and faculty want from KNET. Another slice is things they hate about it.
Slide showing an example of a student completing the Value Proposition Template by Osterwalder et al. There are three boxes - labelled 'Customer Gains', 'Customer Jobs' and 'Customer pains'. Each is filled with bullets that relate KNET to these headings.
Debrief and Discussion
In the debrief, the instructor can encourage students to think about how the issues raised through the analysis just conducted are dealt with in the public context. One example is the issue of diversity: even a small municipality can have a diverse citizenry with different needs (in terms of services, accessibility needs or languages spoken). How should governments prioritize the needs that emerge?
Finally, a key success factor for this class is to be able to engage with users, product managers, designers and developers to conduct cost/benefit analyses. In preparation for this class, David recorded an interview with KNET developers in which they discuss their real world constraints on action and improvement. The purpose of this is to help students to develop empathy for the delivery team, as policy makers and managers may not understand their constraints. Often what seems to be an easy solution might require complex and resource intensive work.
Segment 4 - Organizational Change and Leadership – 20'
Purpose of this segment
In the digital era, being able to identify project contexts with high degrees of uncertainty versus those of low uncertainty is an essential skill for public servants. Despite many bureaucracies' best efforts to do so, it is unlikely that every challenge can be well handled by fitting it into a waterfall-like process. But the reverse is true of agile - not every problem needs an agile-like approach. Indeed, referencing the earlier class on Wardley Maps, an effective public service will take problems with high uncertainty that need agile and, over time, understand them well enough to transform them into low uncertainty waterfall-like issues.
A second key skill will be for managers to convince stakeholders, both decision-makers and team members doing the work, to adopt these new practices. This section highlights the barriers managers might face and seeks to brainstorm some effective practices to overcome them.
Video of David teaching this segment
Examples
To start this section, David shares the example from Eric Reis' book The Lean Start Up. It relates to how management practices at Turbo Tax - a tool that helps filling US taxes - adapted when the software shifted from being sold as CD-ROM to being offered over the web. (A side note: The very existence of Turbo Tax is a symptom of the government failure to make the process easy for users).
When the software was sold as a CD-ROM, the company had to finalize all the features before selling it to customers. As a results any new features or experiments were expensive to test. One had to invest in building them and then wait up to a year before enough data was gathered to see if they were successful. As a result experiments championed by senior executives with clout often won the day.
But when Turbo Tax moved to the web, new features could be launched and tested right away. Indeed, it might only take a few days to gather enough data to learn if a new feature was being used or created value to customers. This lowered the cost of experimentation and allowed more junior staff with creative ideas a chance to test their hypothesis.
The key lesson of this example is that the leadership had to shift. In order for the company to benefit from trying many more ideas it required not only a shift in technology, but also new forms of management and business processes to test so many new ideas. While new technologies may make it easier to test and iterate, management must be willing to drive changes to the businesses culture and processes to allow these new approaches to take hold and flourish.
Three slides. The first two have graphs, the third says "Boy, the amount of learning they get is immense now. And what it does is develop entrepreneurs, because when you have only one test, you don't have entrepreneurs, you have politicians, because you have to sell. So you build up a society of politicians and sales people' - Scott Cook, Intuit CEO, from the Lean Startup by Eric Reis
Discussion - a return the challenges of agile
David then asks students to share stories of when they tried to implement iterative or agile-like approaches in their jobs and what conditions facilitated or prevented them from doing so. The instructor should look for responses that highlight the role of managers. This discussion will surface many of the concerns that managers might have about agile-like approaches. The goal is not to validate these objections but rather to encourage students to think about how they can overcome these objections in the future.
David discusses some practices to overcome these barriers. For example, one common justification for not implementing agile-like approaches is restrictive legislation. Students should be encouraged to dig into these rules. Under the Obama administration, a digital team within the United States investigated this argument and found the law was more flexible than believed. The TechFAR Handbook was a product of this investigation. It shares several practices about how to run agile projects in a manner aligned with existing federal rules and policies. Students and instructors are encouraged to share other examples.
Common questions from students faculty could prepare for:
- How do you determine when to stop doing hypothesis tests?
- How do you explain to policy makers what an iterative and agile approach is?
- What do you do when there is political will for agile approach, but there are not extra resources for that?
How can you get support teaching this unit?
We're dedicated to helping make sure people feel comfortable teaching with these materials.
Send a message to mailbox@teachingpublicservice.digital if you want to book in a call or have any questions.
You can also connect with David on LinkedIn.
What are your rights to use this material?
We have developed these materials as open access teaching materials. We welcome and encourage your re-use of them, and we do not ask for payment. The materials are licensed under a Creative Commons Attribution 4.0 International License.
If you are using any of our syllabus materials, please credit us on your course website using the following text:
We are proud to use the Teaching Public Service in the Digital Age syllabus in our curriculum and teaching. Developed by an international community of more than 20 professors and practitioners, the syllabus is available open-source and free at www.teachingpublicservice.digital
Why was this page created?
This teaching material forms part of the Teaching Public Service in the Digital Age project. Read more about it here.
Acknowledgements
David Eaves would like to note that this material was made possible by numerous practitioners and other faculty who have generously shared stories, pedagogy and their practices. David is also grateful to the students of DPI 662 at the Harvard Kennedy School for enriching the course and providing consent to have the material and questions shared. Finally, an enormous thank you must be given to Beatriz Vasconcellos, who helped assemble and organize the content on this page.