By Whitney Heins
Photography by Brian Notess
There’s no denying that online search engines have become part of our daily lives. We no longer trek to the nearest library and thumb through encyclopedias or printed periodicals to resolve our queries. Today, just a few keystrokes or taps can deliver information to any of our internet-enabled devices immediately.
But have you ever stopped to think about how you pick search words or phrases? What about how you view the results? Do you automatically click the top one or do you scan them all first?
These are the questions that interest Dania Bilal, professor of information sciences at UT. She has spent much of the past 20 years analyzing how search engines operate and how effective—or ineffective—they can be.
Bilal is particularly passionate about making sure middle-school-aged children can find the information they need for class assignments. “I’m interested in this group because they are in an important transition time,” she explained. “If students can develop effective search habits, they will be better prepared for high school. If they don’t, they may fall behind.”
Looking for Answers
In the late 1990s, search engines promising relevance to a younger audience began to spring up along the information superhighway. However, Bilal was skeptical as to whether or not those tools were actually providing more kid-friendly results.
To investigate, she devised a study to measure how successfully kids navigated the popular (but now extinct) search engine Yahooligans! It assessed activities—such as searching, browsing, clicking links, mouse hovering, screen scrolling, backtracking (using the browser’s back button), and looping (repeating searches)—and timed how long it took to complete tasks.
The findings, in Bilal’s words, “were not encouraging.” She believed the trouble was rooted in the way algorithms generated results—they could not duplicate the way people think or provide what the kids expected.
Intrigued, she went back to work in 2000 and designed a more comprehensive study to analyze the use of Yahooligans! by seventh graders. Bilal captured video of the students’ computer screens performing three tasks—finding a fact related to the school curriculum, performing research on a topic related to the curriculum, and seeking information on a topic that interested them. After each session, students were interviewed about their experience.
Bilal’s analysis revealed some navigation behaviors she described as chaotic: Students often backtracked by shifting back and forth between visited sites and looped by repeating the same searches and visiting the same links frequently.
What Google Wants to Know
Flash forward a few years, and many of the original kid-friendly search engines had vanished. “Around 2007 Google began enjoying widespread popularity,” Bilal noted. “People went to Google first—especially children, who went there more frequently than search sites created for them.”
Bilal was curious to see how Google was serving its younger users. “While everyone can access Google, the results are not tailored to different ages or intellectual levels,” she explained.
In 2013, Bilal began investigating the readability of Google’s search results by examining middle school student queries. She compared search results deemed basic, intermediate, or advanced by Google to standard readability scales and found a mismatch.
“Most of the topics searched by the students returned results that were above their reading level,” Bilal said.
The apparent disconnect got Google’s attention. To find out more, the company awarded Bilal and co-principal investigator Jacek Gwizdka, from the University of Texas at Austin, more than $40,000 in 2014 to delve further into the readability issue.
Making the Connection
The Google study asked sixth-, seventh-, and eighth-grade students to perform one reading task and three separate search tasks. Eye-tracking technology was used to capture how the subjects’ eyes moved, where they looked, and how long they fixated on results.
“There is a connection between the eyes and the mind,” Bilal explained. “The eye fixations can reveal their thought processes. We interviewed the students to get a deeper understanding as to why they fixated on certain results and why they clicked on top-ranked results. This is the type of information search engine designers need to look at when presenting results.”
By evaluating the data, the researchers discovered that behaviors differ by grade level. For example, sixth graders seemed to be less discriminating and started by fixating on any result displayed on a page. In contrast, eighth graders started by fixating more frequently on the top results—a behavior similar to that of adults.
“These differences are likely due to how well they understand the concept of results ranks,” Bilal said. “Also, we speculate that users focus on target words that match the description of the search task at hand. It gives us more evidence that variables such as age and reading level can impact how helpful or confusing the search experience can be.”
Generating Ideas
The students were also asked to rate the experience of reading the results. So far, their comments have been very enlightening. “We asked questions like ‘What would you like Google to do to make these results easier to read?’ They recommended things we would have never thought of,” Bilal said.
Some students, for instance, proposed results similar to online reviews or shopping cart suggestions by showing links visited by others who searched for the same topic. Others said results should appear based on credibility, while some craved a more organized design with less distracting text, images, and advertisements.
As Bilal and Gwizdka continue to investigate children’s information seeking and online reading behaviors, they are hopeful that search engines will eventually allow users to indicate their age or reading level (or possibly calculate it) to return more appropriate results.
“We want to encourage search engine designers to build pages that can be personalized,” Bilal added. “Our goal is to build models that could enhance the user experience by accurately predicting a user’s reading level and preferences.”
Considering how much search technology has improved over the past 20 years, it probably won’t be long until a more customizable engine is helping us find what we are looking for.