Background: The Importance of Open Infrastructure
Open digital infrastructure forms the groundwork of the digital world. At a time when personal and professional activity increasingly takes place digitally, the security, reliability and trustworthiness of this infrastructure is of crucial importance. The internet, which runs on this infrastructure, has long provided the public space for the development of a digital society. It is therefore imperative that its foundations are maintained and developed in the public interest.
Although individuals, companies and governments depend upon this foundation of free and public code (or free and open-source software, or FOSS), the way digital infrastructure is developed and maintained has not fundamentally changed since the early 1990s. The development and maintenance of digital infrastructure is still overwhelmingly the work of contributors whose lone efforts can suddenly become fundamental to the successful operation of hundreds of new projects.
In her 2016 report, Roads and Bridges: The Unseen Labor Behind Our Digital Infrastructure, Nadia Eghbal writes: “Digital infrastructure projects […] are conceived of and built from the bottom up. It is akin to a group of citizens getting together and deciding they want to build a bridge or create their own sewage system. There is no authoritative body whose formal permission is required to create new digital infrastructure.” (p.43) “Many useful projects will come from independent developers who suddenly find themselves at the helm of a successful project, facing critical decisions about its future.” (p. 52)
This report looks into the kinds of support needed by the individuals and groups involved in the development of digital infrastructure, and examines why existing support such as project-based fellowships and development grants are rarely used. It builds on the work of others, in particular Nadia Eghbal, whose Roads and Bridges formed the basis of for the call for research on digital infrastructure by the Ford Foundation. This report was produced by Implicit Development Environments (IDE), a project supported by the Ford Foundation in the context of that call.
Problem Statement: Reliance Without Governance?
Open digital infrastructure is not just part of the digital ecosystem; it forms the very foundations of the technology industry. Nadia Eghbal writes: “Thanks to permissive licenses, companies like Facebook or Instagram are not obligated to pay for this code [i.e. free, public software], but are free to profit handsomely from it. This is not unlike a trucking company (Instagram) using a highway (public code) to transport goods for commercial services (Instagram’s app).” (Roads and Bridges, p. 20)
Like roads and highways, digital infrastructure is more than a backbone for the economy. Like market squares and other public spaces, it also serves as an arbiter of society. Digital infrastructure grants access to information, makes and shapes personal and professional digital environments, and enables the formation of communities (or, conversely, when it is not built with the public interest in mind, doesn’t).
Where open infrastructure is absent, privatized and often exploitative business models predominate. These commodify people, harming those communities that are most vulnerable, both in the physical and the digital world. While the existence of public infrastructure does not automatically make the world a more just and equitable place, it is an essential prerequisite.
Contributors to public infrastructure include public institutions (e.g. public wireless networks), businesses (e.g. IBM-owned RedHat’s platform and cloud services), non-profits (e.g. the Internet Security Research Group service Let’s encrypt), and often a combination of all three (e.g. DNS root servers). Nadia Eghbal classifies the development of digital infrastructure according to three main categories, each of which is differently organized and financed: within companies, as a new business, or by an individual or community of developers (Roads and Bridges, p. 46). This report focuses on the third group for the following reasons:
- Representatives of this group are comparatively hard to identify because organizational structure, public image and branding are often absent. Characteristically, their working methods are decentralized, with a focus on individual contributions and self-organization rather than shared strategies or missions.
- While many of the projects produced by this third group are well known in the tech industry, the people and communities behind them often fly under the radar of investors, funders and other non-technical supporters. As a result, they receive less support.
- Even though we currently do not have the data to quantify how many digital infrastructure projects rest mainly or solely on the shoulders of individuals or developer communities, anecdotal evidence suggests they make up a sizeable share of these projects. This is to some degree supported by a 2015 study by Guilherme Avelino, Marco Tulio Valente and Andre Hora which showed that of the 133 most widely used projects on GitHub, 64% are maintained by one or two contributors. Existing studies focus on projects that are on GitHub and evaluate their contributor numbers. This approach provides many valuable insights about the FOSS ecosystem, but doesn’t cover two aspects that are of interest to this report: Whether or not these projects have institutional support, and projects that work with other tools than GitHub.
For this report, we chose to incorporate an additional actor missing from Eghbal’s classification into this third group: not-for-profits. A great deal of infrastructure is developed by communities made up partly by paid staff, but mostly volunteers and freelancers, often under the umbrella of not-for-profit organizations. Compared to for-profits, not-for-profits find it easier to rally these communities, who will frequently volunteer their time and expertise. Even though not-for-profits constitute an organizational structure that is otherwise absent from the cited developer communities, our research has found them to play a vital role both in the digital infrastructure ecosystem and as a counterweight to businesses and commercial interests.
To continue the metaphor of digital infrastructure as roads and highways, this means that road workers carry out their jobs in the absence of any form of central governance. They proactively maintain old roads and continuously extend the road network on which not only trucks carrying companies’ loads travel, but along which public transport is conducted – allowing people to go about their lives and reach their places of work and study.
This infrastructure is of increasing social and economic importance, and it deserves support. Major donors to public-interest digital infrastructure include private foundations (e.g. Mozilla Foundation, NLnet Foundation, Knight Foundation), public funders (e.g. the Open Technology Fund, the Prototype Fund, the Next Generation Internet grants of the EU), and multi-stakeholder initiatives (e.g. the Core Infrastructure Initiative). Private individuals contribute via direct donation and crowd funding platforms (e.g. Liberapay, Open Collective) or platforms like Bountysource, on which donors can sign up to support individual features under development.
Yet it is doubtful that these sums are enough. Whether public infrastructure should be supported by philanthropy is another subject for debate. The fact is however, that infrastructure projects developed by individuals or developer communities continuously face the challenge of how to support themselves.
Goals: How to Sustain Open Infrastructure
“Developing effective support strategies requires a nuanced understanding of the open source culture that characterizes so much of our digital infrastructure, as well as recognizing that much has changed in the past five years, including the very definition of ‘open source’ itself.” (Roads and Bridges, p. 125)
In order to better understand the culture from which open digital infrastructure emerges and the obstacles faced by different projects, we sought to identify similarities and differences between FOSS projects. Through a series of qualitative interviews, our goal was to identify the social norms and practices that influence contributors to infrastructure projects, and thereby understand the effects of these values on open source communities – particularly with regards to funding. The goals of our research were to:
- identify the preconditions and needs specific to infrastructure projects which set them apart from projects at the application layer,
- examine in what way these needs are rooted in the social values underlying free and open-source software (FOSS) communities,
- outline how funders can support open digital infrastructure in the future.
As we have noted, open digital infrastructure projects tend to be organized from the bottom up by their contributors. We therefore started our research right where the work is done – with those individuals. We conducted a total of 26 interviews, 15 of which focused on the interviewee’s relationship to a particular project (across 12 projects), while the remaining 11 focused on the relationship between the interviewee and communities within FOSS.
We opted for a qualitative, human-centered research (HCR) approach. HCR, which has been described as the “active consultation of people”, can be especially valuable in cases where the factors at play are unknown or not well understood, as it helps with mapping the field and allows any further research, qualitative or quantitative, to build on an accurate understanding of the context. For our research, HCR helped us to dive into the subject matter early, adjust our strategy on the go and remain flexible in the paths we pursued, while remaining anchored in the communities we aimed to study.
Together with project partner Simply Secure, we formulated a large set of questions to assist interviewers in navigating interviews. In turn, interviewees chose for themselves which aspects of their work they wanted to emphasize and fed back to interviewers with suggestions of questions to add to the list.
The questions fell into 7 broad categories:
- Trajectories and positions in FOSS: What are you working on, and how did you end up there?
- Outlook on FOSS: What do you like about your work?
- Values: What does your work on open infrastructure mean to you?
- Governance: How do people on a project collaborate? Who does what, and why?
- Support: From what sources do projects receive funding or sponsorship?
- Project state and outlook: Is your project sustainable? What makes it so?
- Standards: How do standards affect your work, and how does your work affect them?
In their 2019 report On Trust and Transparency, which was also based on a series of interviews, Simply Secure highlighted how the power differential between funders and grantees may have unknown, and sometimes undesirable, effects on funding and the relationships between parties. Following the example of Simply Secure, we placed great emphasis on confidentiality and trust in the course of conducting and evaluating interviews. Part of this effort consisted in running our own open-source and GDPR-compliant digital research infrastructure. No individual interview partner or project will be named in this report without their explicit consent.
What is (open) digital infrastructure?
What qualifies as digital infrastructure is open to interpretation. Definitions range from the formulation and implementation of information technology standards, to the tools and services that enable developers to do their work. For us, neither definition by itself seemed promising. Standards laid down by organizations such as the Internet Engineering Task Force (IETF) do not always get accepted in the wilder space of open-source software development, where quasi-standards may rule over official ones. And to single out specific tools, frameworks and services seemed arbitrary: why would we assume that special rules apply to them that do not apply to other examples of free or open-source software?
Instead, we formed our own working definition to match our research question. Digital infrastructure provides services essential to the operation of digital activity. It is the result of processes that take place in different fields and are shaped by many different people in varying capacities. Like our research methodology, our definition of infrastructure puts the people that work on it at the center. We identified three core spheres of activity and the actors within them:
- Standard creators People and organizations who decide upon and write IT standards.
- Standard implementors People who implement these standards in their software and create digital tools and services around them.
- Service providers and maintainers People and organizations who run these tools and services, and can directly observe how these standards affect users.
If we want to assess how open digital infrastructure is best developed and how funders can support it better, we needed to address these three groups equally, viewing them as an interconnected ecosystem.
From the vast number of existing infrastructure projects, we intentionally focused on developers with strong ties to infrastructure security or resilience. Though they make up a sizable community, they tend to set themselves apart from more public groups working on frameworks and languages. Even when their work is publicly known, they are therefore less visible, despite their work playing a vital role in making digital systems – the network of roads and highways – more secure. Since most of these projects value privacy and decentralization, they tend not to share their code on GitHub, but through other, decentralized channels such as GitLab, Bitbucket and Sourceforge. Most research has focused on GitHub because its centralized structure enables researchers to query data on large numbers of projects. However, this has led to projects present on other channels not being represented in studies into open digital infrastructure.
3 Interviews: Who did we talk to?
For our interviews, we contacted people contributing to different kinds of open digital infrastructure projects through a variety of roles and tasks. Most of the time, we contacted them directly or were introduced by a former interviewee. We also attended different communities’ events in order to schedule interviews in person; this proved more effective. The demographic of our interviewees was rather homogenous – mostly male-presenting and overwhelming white or white-passing. This was in line with our observations at events as well as an empirical study on the geographic locations of open-source software developers on GitHub conducted by Yuri Takhteyev and Andrew Hilts. In an effort to counter this homogeneity, we invited additional projects with ties to under-represented groups and world regions to participate in this research, but received no responses.
Interviews in Numbers
We conducted 26 interviews in total, 25 of them in person, one via telephone. 15 were in-depth interviews with people from 12 different projects, 11 focused especially on the relationship between individual and community.
Challenges and Limitations
A few challenges we faced during our research are worth noting. Because our target communities value security and privacy highly, we mostly reached people close to our own network or to whom we were introduced by a trusted person. We tried mitigating this by sending out a public call for interviews and requesting interviews on public mailing lists, but received few responses.
For trust and privacy reasons, we limited our set of demographic questions to a minimum. Nevertheless, most interviewees opted out of stating even basic information like gender. There may be several reasons that led to this question going unanswered, but one interviewee stated that they perceived it as “politically loaded”. The only vaguely-reliable numbers about women in Open Source (which concluded female representation to comprise 5.4%) are based on GitHub profiles, a platform that many in our target audience shun for different reasons (e.g. it is a centralized, closed-source service belonging to the Microsoft corporation). Even more, the GitHub numbers are likely to be skewed, since pull requests by observably female profiles are more often rejected. This compounds the problem of finding reliable statistics with which to compare the demographics of our interview group.
4 Project Types
We matched statements from each individual interviewee to the actual infrastructure project. Since many interviewees were involved in more than one project, we asked them to choose the one in which they were then most involved, since they would have a thorough understanding of how the community works. To make the commonalities and differences between the projects more tangible, we formulated “project types”. These project types help provide an overview of the landscape, helping us keep the full breadth of the field in mind as we analyze our interviews. The types are of course simplified – by no means do all infrastructure projects fit neatly into one category, nor do they check all the boxes. Mainly, they serve as a tool for differentiating insights and recommendations that might otherwise seem conflicting or contradictory.
The “one-person shop”
The “one-person shop”, as one interview partner described it, is the smallest unit: individual people who work on an independent project for which they are solely responsible.
- Who is involved A freelancer or volunteer.
- Longevity A one-person shop can exist for decades.
- Pipeline More likely to work on standards or implementation than running services.
- Funding No first-hand experience.
- Resources The limiting factor is time. A one-person shop is often cross-financed through contracts or regular employment (e.g. a position in academia).
- Governance No structure needed.
- Note Even though they function independently, the one-person shop is very well connected within the wider sector. The willingness to collaborate varies.
Collectives are grassroots communities whose contributor numbers fluctuate. Consensus on general issues is generated through conversations on IRC channels, mailing lists or, less often, a platform like Slack or Matrix; smaller decisions can be taken by those who work on the specific issue.
- Who is involved Freelancers who usually offer services connected to the project, and volunteers.
- Longevity Collectives are persistent, even though contributor numbers fluctuate and the original founders may no longer be involved.
- Pipeline Implementation and services.
- Funding Little to no first-hand experience, often with a critical or skeptical attitude towards funding.
- Resources The project has no funds; if it does, distributing them is seen as a challenge.
- Governance Consent-based decision making, meritocratic; often anti-structure and/or anti-hierarchical. No set roles apart from core developer and contributor.
- Size Anything from two contributors to dozens or even hundreds.
- Note Personal and political values tend to be more explicit in collectives’ work than the other types. Despite producing a lot of work, there is usually is no identifiable spokesperson (e.g. for companies who are looking for someone to address in order to collaborate; for funders looking for a contact person).
The embedded project
Embedded projects typically evolve out of an older or larger project (quite possibly a collective) and specialize on developing a set of features within that project – for instance, a product or a service. Embedded projects have strong ties to the projects from which they sprang as well as other specialized projects, and often share resources and distribute tasks between them. Embedded projects function within a network.
- Who is involved From an initial group of freelancers, a core of paid staff is formed who push development forward; freelancers who offer services connected to the project; volunteers.
- Longevity An embedded project can evolve out of older, larger tech projects (quite possibly a collective) and can specialize on a set of features or a product.
- Pipeline Standards, implementation, and services.
- Funding Embedded projects> often have first-hand experience applying for support from different sources (e.g. paid contributions from companies, FOSS foundations, philanthropic or public funders), but less often with handling larger grants.
- Resources Embedded projects have modest to mid-size funds at their disposal which mostly go towards supporting coding work, but can also fund travel, community events etc.
- Governance A light-weight governance structure is in place, but mostly to coordinate with external partners. Internal roles, such as fundraisers and community managers, are vaguely defined, if at all.
- Size Small team of 3-10 people.
- Legal status An embedded project starts without a legal entity, but soon adopts a low-cost model to suit its needs, which might include writing bills, handling money and partner contracts, insurance reasons, and so as to be taken seriously by companies and potential funders.
- Note Embedded projects are constantly evolving and flexible, which sets them apart from organizations.
Organizations have transformed their product into a brand with which they are inseparably connected.
- Who is involved Paid staff with clear job descriptions. Usually a community of freelancers and volunteers who contribute a lot to the product forms around the core staff, though often they only play a minor role.
- Longevity The organization is established.
- Pipeline More likely to work on implementation and services than standards.
- Funding The organization has received funding from different donors and other sources. They usually have a mixed funding model consisting of contracts, services and grants.
- Resources A significant level of resources goes into maintaining the organization itself, i.e. in management, marketing and overheads.
- Governance Organizations have typical structures with clearly defined internal (community management, team lead, CEO) and external roles (marketing, fundraising, press).
- Size Small to medium-sized team of 5–30 people.
- Legal status The organization often has, or tries to achieve, a more sophisticated legal status, such as an LLC, or a fiscal status that allows for tax-exemption. Depending on the country or region in which the organization is based, the latter can take very different forms and have very different strings attached.
- Note It is possible for an organization to take other projects (e.g. young embedded projects) under its umbrella, providing them with legal and fiscal status. However, there needs to be some sort of logical connection between the two – organizations do not usually serve as a fiscal sponsor in the classical sense.
Open digital infrastructure is overwhelmingly developed and maintained by individuals or groups of contributors working in the public interest. It provides the foundation of all digital technology. As one of the pillars of modern society and communication, open digital infrastructure deserves support from funders. Public and private funders provide a large share of support for FOSS projects, but to support this work efficiently, funders need to understand these projects’ strengths as well as their common challenges. By working together with grantees, funders can identify how to diversify infrastructure communities and create more stable and less privileged working conditions without weakening the core values of decentralization, self-organization and intrinsic motivation which drive FOSS development.
For the Internet to provide the public space necessary for an equal digital society, it needs to be more than just roads and bridges. For this to happen, open digital infrastructure and the people who build and maintain this software must become more visible, more understood, and more appreciated.
In general software development terminology, adoption describes the rate at which users change to another technical system because it better answers their needs. Infrastructure code however, is not usually adopted directly by users but by other software projects, meaning adoption rates are indirect and more difficult to assess. Adoption is often used as a metric of success for software products.
The abbreviation for free and open-source software (i.e. software that is published under a free or open license, is human-readable and can be lawfully copied, edited and developed further).
The General Data Protection Regulation is a binding framework for data protection laws within EU member states, where data for this report was gathered. All tools used in this research were GDPR compliant.
GitHub is both the name of a widely-used platform and the company behind it. The centralized platform uses the open-source version-control software Git and hosts the code base of many FOSS projects, while parts of its own software are closed-source.
HRPC Research Group
The Human Rights Protocol Considerations Research Group is part of the Internet Research Task Force (IRTF). It conducts research on how standards and protocols impact human rights, and comments on standardization discussions.
The Internet Engineering Task Force is an organization that develops Internet standards. In principle, anyone can participate in its working groups or attend its meetings. An informal slogan of the IETF is “rough consensus and running code”, which describes its method for decision making on the basis of working systems.
The Internet Relay Chat is a protocol for text-based communication that was developed in the 1980s and is still widely used in FOSS communities. Alternatives include more modern protocols like Matrix, or platforms like Slack.
The Internet Research Task Force complements the work of the IETF in that it promotes continuous, more perpetual research on Internet standards.
The following questions served as a guideline during our interviews and were continuously adapted according to the interview situation, the interviewee and the project.
- What name do you go by?
- How do you identify your gender?
- How would you describe what you do right now?
Your trajectory and position in FOSS
- Why did you get into OS development?
- What is your educational background?
- What projects are you most involved in?
- For those projects: What is your role?
- How long have you been doing that?
- How did you get involved?
- Are you part of a team, and if so, how big is it?
- Have you had major involvement with other FOSS projects in the past? Which projects, and in what capacity?
- How does your FOSS work interact with your day job? Is the balance where you would like it?
- How are you supporting yourself, if not by working on the project?
Your outlook on FOSS
- What is your favorite part of your work?
- What’s your least favorite part of your work?
- What do you perceive as the most important FOSS digital infrastructure projects out there? Why?
Governance & Management
- What are the communities that you would describe yourself as being a part of?
- How did the last person join your project?
- How did the last person leave your project?
- How do you define “contributor” to your project? Where do your contributors come from?
- Is there anybody who isn’t coding who is a member of your core team?
- Who is not on your team/community, but should be?
- Are there structures about decision-making in your project/area?
- What was the biggest challenge in your current project/area?
- Can you tell me about a time that a conflict occurred in your project? What happened?
- How do you share knowledge in your project/area?
- Do you have a mentor? Do you see yourself as a mentor?
- Have you ever met people on your project face-to-face?
- What conferences do you attend for your work?
- From where does your project get its funding and support?
- What kind of non-monetary support does the project get?
- Do you need more support? If so, for what?
- Is there anybody you wouldn’t accept support from? Why not?
- Have you ever applied for or received funding? How did that work out for you?
Standards – for standards people
- Did you ever comment or write up an RFC? How did that work for you?
- Do you talk to people who are using protocols and standards to which you are contributing? Is there a feedback loop?
Standards – for developers
- How does standardization impact your project (if applicable)?
- What other projects or code bases does your project rely on?
- Do you provide feedback on standards or contribute to discussions about them?
For implementers and service providers
- What are some roadblocks for you when implementing code?
- How does standardization impact your project, if applicable?
- Do you ever give feedback about standards? To whom?
- What other projects or code bases does your project rely on?
- Do you see your work as political? Why/why not?
- Do you see yourself as an activist? Why/why not?
- How do you describe your FOSS work – is it a job, a hobby…?
- What does “open” mean to you? In what ways are your projects open, in what ways less so?
- Do you keep track of your time worked on FOSS?
- What keeps you up at night?
Your project’s state & outlook
- What does your work mean to you?
- What effect do you think your work has on society?
- What do you wish more people knew about your work?
- Do you think your project/area is sustainable? What are your role models for sustainable FOSS infrastructure?
- What is your best-case scenario for your project/area in 5 years? What is your worst-case scenario?