Exploration Engines

A Fix for the Feed?

You’re receiving this email because you’re signed up to the koodos newsletter.

koodos’ collective (inspired by Andy Warhol’s Factory) is a selective group of young talented engineers & creatives that sit adjacent to the core koodos team. The collective designs & builds stand-alone open-source product experiments that contribute to a creative layer on internet content. Below is an experiment from the koodos collective.


Serendipitous use of the internet is slowly going extinct as we replace link-hopping with the algorithmic-feed. Ranked results and recommendations have become the dominant mode of exploring information online. In this experiment, we break away from this paradigm, and present Wikigraph - our project for Interhackt. While a “search engine” returns a ranked list of results, Wikigraph returns the most relevant sub-graph of pages. Such an application we term an “exploration engine.”

Check out the experiment here: Giantgra.ph

Share

The introduction of the “feed” in modern social media has been controversial. At its core, “algorithmic curation is ultimately about power — about who controls what we see and how we see it.” As our lawmakers now discuss this question on Capitol Hill, most of the discussion is centered around how we can “fix” the feed. However, our political and technological leaders still take the feed as a given. This piece suggests an alternative: rather than ranking content algorithmically, we propose an “exploration engine” which returns a sub-graph of linked content.

Wikigraph is the first (usable) exploration engine for searching connected subgraphs of Wikipedia pages. Applying this idea to other media platforms is an alternative to our current search and recommendation technology.

Motivation

A common criticism of visualization technology is that it doesn’t enhance productivity. Some (we) have labelled these products as “YACVTTDRDMFTU” (Yet Another Cool Visualization Technology That Doesn’t Really Do Much For The User). However, smart UX and good graph algorithms empower the user to find content and make connections more effectively than traditional search engines and feeds. The big difference is that while they are optimized for finding particular pieces of content — an exploration engine is optimized for analyzing large communities of content.

Looking at content at the community scale allows for four main types of insights:

  1. Unexpected community. Discovering a community containing known content not expected to be interrelated.

  2. Unexpected membership. Discovering an unexpected or unknown member in a known community of content.

  3. Unexpected connection. Discovering an unexpected connection between known content across communities.

  4. Unexpected degree. Discovering a known or unknown piece of content with an unexpectedly high degree.

Wikigraph uses the simplest possible algorithm to generate graphs, which is particularly good for making unexpected connection discoveries.

It works as follows: the algorithm picks the most relevant page as a starting point, and follows all outgoing links, moving in concentric circles. Wikigraph performs this twice, meaning two concentric circles out from the original page. We then apply a random filter to the results in order to prevent the “hairball” problem [Fig. 1]. This strategy usually leads to “hub and spoke” structured graphs [Fig. 2], which are good for discovering unexpected links between pages.

Fig 1. A preliminary test of Wikigraph, with no filtering. Since each Wikipedia page has on the order of hundreds of links, our algorithm produces a link structure so dense that it is completely useless — and it looks like a hairball

Soon, the technology will be viable for much larger graphs, on the scale of Facebook, Twitter, and YouTube. Rather than a random filter, we can use advanced link-cutting strategies to make large graphs usable. The implication is that if we are unhappy with recommendations, we can easily “zoom out” to find different perspectives. We can always “zoom in” elsewhere to find different content. Browsing through content on any platform will feel like Google Maps.

Fig 2. After applying filtering to the graph, we get “hub and spoke” structures — which have a single central node and somewhat limited connections between lower-level topics

Creative Machines

Beyond commercial applications, exploration engines have the potential to enhance our creativity. If Steve Jobs is correct that “creativity is just connecting things,” then Wikigraph is a creative machine — a bicycle for the mind. We can literally see the missing link, and find the disparate connection. It can take us to the frontiers of knowledge: where the links, nodes, and cliques are sparse.

The history of technology and globalization are the expansion and connection of knowledge. All written knowledge, from the Rosetta Stone, to the Talmud, to the modern academic paper, are graph structured [Fig. 3]. Yet, 21st century digital media still imitates the linear scroll. It’s time to update how we create, search, and explore information. The feed is an effective interface for presenting content, but it diminishes human agency and curiosity, and therefore also our understanding, discovery, and creativity. With Wikigraph, the first experiment from the Koodos Collective, we hope to inspire new products and interfaces that don’t lease our taste to the private manipulation of algorithms, but one where users are in control. One where we are free to explore.

Drew Tada & Pam Beardsell