Five Things I Have Learned from Information Systems

It is not the amount of knowledge that makes a brain. It is not even the distribution of knowledge. It is the interconnectedness. ― James Gleick, The Information

While researching next week’s topic (emergence), I came across the Five Things I’ve Learned website, a collection of talks including Five Things I’ve Learned About The Art of Brevity or Being Grumpy or Leonard Cohen or Writing Fiction (from Isabel Allende). I love lists; I am a sucker for list-making writing prompts. I stopped researching and answered the call: After nearly 20 years, what are five things I’ve learned about (information) systems?

Not the things I needed to learn to build them, like CMS software, Vue, Go or Kafka … the things the systems themselves have taught me. Here’s what I came up with …

The mental (and emotional) ecology of an organization architects their information systems and defines what’s possible.

Many challenges have required a constant technology skillset upgrade, but my ability to do excellent work depends more on my thinking and communication skills. I often quote Conway’s law because every situation has taught me that he was correct. The system’s architecture will mirror the communication structure, process and attitude of the organization.

Systems are designed by human interaction. People in silos build technology silos. People who get along well push code into production that gets along well with its environment. When there is a chasm between “product” and “tech” people, the technology system is perpetually unsatisfying because it can’t bridge that gap.

Over my career, the systems that have evolved gracefully and deftly have been the ones supported by intellectually-vibrant and emotionally-respectful human interactions. Improving the mental and emotional ecology improves our ability to deliver evolving systems in the midst of uncertainty.

When we are mentally and emotionally defensive, we build defensive systems of software that communicate poorly. This is clearly a problem for an “information” system whose entire purpose is communication.

What we prioritize – usually isn’t healthy for the system.

The organizational approach to most projects is what I call feature-driven engineering (aka requirements-driven engineering). “Add this feature to the software, based on these requirements.” People managing the feature’s budget are reluctant to “waste” their money building capabilities – shared functionality that can be used across the system. Improving and expanding capabilities is labeled “clearing up tech debt”, paying to clean up other people’s mess. It takes too long, I’ve heard many times, to change the software, we need to work around it. Even though the tech debt arose from everyone else making that same choice.

Over time, the code base becomes a loose nation of factions that represent the structures of budgetary power and influence within the organization. The prioritized features were the ones with the most budget allocated to them but may not have been what the system most needed. And might not be what is needed to grow into the future.

Features are rarely “finished” – ideas are infinite but budget is not. So the system lives in a constant, uneasy state of “hoping some change doesn’t set the whole thing on fire” and “maybe someday we’ll refactor.” This defensive posture becomes gatekeeping and the code begins to calcify.

Our priorities when hiring are similarly unhealthy for the system. In the same mechanistic way, we interview people who fit our “requirements” and can deliver what we already have in production. Rather than hire people who easily learn new things, adapt as circumstances change and think well with others. To attract and retain systems people, we need to prioritize thinking and communication structures, people systems, that support their ability to deliver a system, together.

I respect the spirit of product-first thinking. A specific type of intelligence is needed to interweave people’s experience with business and tech realities — the humanistic middle ground between revenue generation and 99.999% debates. Tech people, myself included, over-focus (constantly) on the tech. Arguing for Kubernetes with no interest in how the tool will positively impact people who use the system or why the financial investment makes the system as a whole more valuable to the organization.

Simultaneously, product-first approaches have delivered some really crappy software. Refactoring doesn’t fix it because the thinking is baked into the organization. Product thinking is a key part of systems design … but it is not, in itself, systems design.

Counterintuitively, a systems approach delivers more of what everyone (truly) needs, faster. Healthy patterns and relationships in a purpose-driven system are inherently emergent. They open up potential for future products as well as serve the current ones. We can constantly evolve these patterns, rather than (only) create features in software.

Today I am more convinced than ever. Conceptual integrity is central to product quality. Having a system architect is the most important single step toward conceptual integrity. – Frederick P. Brooks Jr, Mythical Man Month: Essays on Software Engineering

Information systems don’t want to be contained and controlled, they want to be well-structured and ubiquitous.

Before the digital era, information (in books, newspapers, letters, magazines, encyclopedias) was shared on pages. Our foundational data object, defining the boundaries and structure of information, was closely tied to paper documents. The internet was invented to markup and share academic papers. Discussions were message boards.

A blog post is a digital article. Magazines are subscription websites. Software created boundaries and stored information objects to be delivered like a collection of pages. Social media is message boards at scale.

Now, information is increasingly interconnected and interdependent. We consume the same information in a different context, which changes what we want to know. For example, when browsing a restaurant on my desktop, I want to see the menu. On my phone, four blocks away, I want directions. Information is no longer static, it’s a dynamic interaction dependent on context.

Much of our information is trapped in “legacy” software whose boundaries block these emerging and dynamic relationships. The key to digital transformation is designing well-structured information flows that adapt and change in relationship to software. We need to view information as being in motion, a choreography, rather than fields in a MySQL database.

The primary blocker is: we still don’t know what the business model for ubiquitous information looks like.

Designing relationships is what matters most

The flow of necessary, helpful and timely information, between people and in a technology system, is the hallmark of good relationships. Good relationships evolve and adapt as circumstances change, while also effectively and efficiently meeting the needs of now. Like a road trip, where you know where you are going but are also enjoying the journey.

Whenever we demand “concrete” workflows, we are cementing what we want right now by denying the potential for growth long term. People, and systems, are not industrial. They are a collection of evolving relationships. While a system may be complex and uncertain, its relationships can be well-designed and reliable.

When we talk about digital transformation or modernizing, we primarily mean (from the system’s point of view) improving the relationships in the system by creating healthy communication patterns. Relational patterns are the heart and soul of information systems … and the people systems that deliver them.

Information is not knowledge

Information is what we consume from sources – the text in a news article or a book, the “how to” in a Youtube video. Knowledge is the act of forming relationships between pieces of information. We mature these relationships by synthesizing information with experience, sound judgment and a deep understanding of the circumstances. Knowledge is the ability to form new relationships between information as circumstances change.

The word knowledge is often used to describe information. Information is a recipe; knowledge is a deep-enough understanding of ingredients to cook improvisationally.

Information is not knowledge and knowledge is not wisdom. – James Gleick, The Information

In the digital world, we’ve set information in motion, where it interrelates and impacts people, like the weather. Knowledge, as I’m defining it, is socially constructed. It’s the meaning we make (together) from the information we gather in the situations we are experiencing. If we don’t think critically, and systematically, about these relationships, they can be hijacked to weaponize opinions. Information dissociated from knowledge can be a weapon of mass destruction.

Software itself is knowledge. It represents the ways we’ve synthesized disparate information, experience and judgment. When we architect information systems, we are designing sociotechnical schemas that create relationships with other information in meaningful, relevant ways. This is exciting stuff. We might be architecting future knowledge.

​The Information: A History, a Theory, a Flood

by James Gleick

“An eye-opening vision of how our relationship to information has transformed the very nature of human consciousness. A fascinating intellectual journey through the history of communication and information.”