A wordcloud of words I’ve used while Twittering…

Dave Davison at Thoughts Illustrated gives an interesting illustration of the “filter on the way out” idea from EiM. He notices that Picasa has been silently aggregating the images in his blog. Now he can go back through it and notice relationships and trends. He gives six steps of working through the pile of images. The six step certainly seem to work for this example, but I’m not sure how generalizable it is since, in my view, the miscellaneous is a pile of raw potential for the emergence of every sort of understanding and meaning, from noticing that you’ve used lots of pictures of Michael Jackson to running statistical and semantic tools that discover deeply hidden relationships.

The idea behind “filter on the way out” is that it’s often (usually?) better to give users tools for sorting through the pile the way that suits them than to <i>only</i> give them a single, pre-baked categorization.

Thomas Mann (no, not that one) has a fascinating and important article about why tagging, folksonomies, and the rest of the hip Web 2.0 stuff is inadequate to meet the needs of scholars looking for information. It is, at least informally, a response to the Calhoun Report.

His example of trying to find information about “tribute payments in the Peloponnesian War” is classic and convincing: Finding what the scholar needs requires smart human guides and the smart guides that humans have created for scholars.

But, of course that doesn’t scale:

I would be the first to agree that the inexpensive indexing methods of term weighting, tagging, and folksonomy referrals–none of which requires expensive professional input–are entirely appropriate for dealing with most of the Internet’s Web offerings. With billions of sites to be indexed, it is out of the question to think that traditional cataloging can be applied to all of them. No one in his right mind would say otherwise.

But there is a crucial distinction that is being swept under the rug: the difference between quick information seeking and scholarship.

And, he says, scholarship requires books. Thus, the labor- and intelligence-intensive scholarly information clustering techniques will continue to work because the flow of books will continue to be relatively slow:

The universe of books published every year is much smaller, and much more manageable, than the universe of Web sites; this is the “niche” of sources to which professional cataloging should be primarily devoted. … Most of the billions of Web sites do not merit this level of attention to begin with; they are too inconsequential and too ephemeral. If we are going to promote scholarship, it is not enough to simply digitize the books for immediate retrieval if term weighting of keywords, tagging, and folksonomy referrals are the only mechanisms we provide for finding them. It is not at all unrealistic to propose that research libraries fill the niche of providing the best, most systematic, access to books…

He later says that systematic cataloging should not exclude all non-books.

As an argument for maintaining human expertise in manually assembling information into meaningful relationships, this paper is convincing. But it rests on supposing that books will continue to be the locus of worthwhile scholarly information. Suppose more and more scholars move onto the Web and do their thinking in public, in conversation with other scholars? Suppose the Web enables scholarship to outstrip the librarians? Manual assemblages of knowledge would retain their value, but they would no longer provide the authoritative guide. Then we will have either of two results: We will have to rely on “‘lowest common denominator’”and ‘one search box/one size fits all’ searching that positively undermines the

requirements of scholarly research”…or we will have to innovate to address the distinct needs of scholars.

My money is on the latter.

He concludes:

We need to make the best possible use of our principles, our experience, our tested practices, and our technologies, and not yield to the temptations to let either the technologies themselves or transient fashions constrict our vision of what needs to be done to promote scholarship of the highest possible quality–and that is a goal very different from striving to provide ’something quickly.’

Amen.

(Thanks to Bradley Allen for the link.)

[Tags: thomas_mann tagging catalogs library_of_congress libraries taxonomy folksonomy everything_is_miscellaneous ]

Atif Rafiq, founder of Bazooked, takes the miscellaneous as his starting point for reflections on differences in how users selection information “depending upon where they stand in the consumption lifecycle.” He writes:

As a user begins to get deeper into consumption, choice begins to multiply not because there is any more content out there. When users interact with information, their interests can grow, branch or re-formulate. Initial inspirations may still apply but new ones are brought to light. In other words, interaction grows the possibilities in the mind of the user.

Then the question is how to deal with the user’s need at these different phases. But that’s what Atif’s post is about…

The always enjoyable Andrew Hinton has an insightful, witty, surprising set of slides ‘n’ text that tries to explain not only what Information Architecture is, but why it’s been so hard to explain. Along the way he has things to say about communities vs. communities of practice, how to attract flies, and why Wikipedia is more like an AK-47 than like an M-16. Great stuff, entertainingly and elegantly communicated. [Tags: information_architecture andrew_hinton ]

Well, not exactly. Siderean has announced that a pilot deployment for Elsevier has over one billion RDF triples (the press release says “relations,” but I assume that’s what that means) in what Siderean calls a “relational navigation” system, i.e., a faceted system that allows for looser links across and among the resources.

I’m working off a press release, so I’m probably getting some or all of this wrong. But, it’s still a heck of a lot of relationships. [Tags: siderean faceted_classification everything_is_miscellaneous rdf]

Technorati has released a widget that displays of the top tags used on a particular blog. Here it is for this blog:

View blog top tags for this blog

And here’s one for my other blog, JohoTheBlog:

View blog top tags for Joho

(Disclosure: I’m on Technorati’s board of advisors.)

EngineeringVillage.org has about 32 million records available, including 10.7 million from the Compendex (Computerized Engineering Index) that has data going back to 1884, 9.5 million records from the Inspec Archives that goes back to 1896, 2.2 milllion government technical records in the NTIS collection, and 9.5 million patent abstracts.

How can you possibly navigate 32 million records? Searching requires second-guessing authors, and with that many records, it’s bound to miss more than it finds. So, EV uses a combination of full text searching and faceted navigation.

For example, if you’re looking for anti-gravity devices, begin by doing a text search on “gravity.” You’ll get 202,162 results. In the righthand frame, you are shown eight areas (facets) — source, author, affiliation, country, document type, year, etc. — each with a list of the occupants of that particular branch. So, under Affiliation, you can see that the Jet Propulsion Lab has 326 records that contain the word “gravity,” while NASA’s Goddard Center only has 155; this by itself is valuable information. Check the NASA box, and now you you can further refine the 234 results by deciding only to see those articles published in the US, and then the ones on solid state physics. We’re now down to 11 articles. But we can always go back and remove the restriction to only articles published by NASA. It’s tree browsing where we get to construct the tree.

Now EngineeringVillage has added user-created tags. Tags can be declared as public, institutional, or belonging to a user-defined group. Very cool. (It would be especially helpful if, say, the US Patent Office were to suck in the tags applied to patents.)

The tag cloud shows that the top tags at the moment — early days for the tagging feature — are “Thermal management,” “sathya,” “Unsaturated soils,” “Wireless sensor networks,” “Photonic crystals,” and “Room temperature,” which suggests that users are working on growing photonic crystals at room temperature for use in wireless sensor networks, to enable the Sathya Sai Organization at long last to achieve world domination.

In an email, Rafael Sidi, VP of product engineering at Elsevier Engineering Information says that the faceted system was built in house using the FAST search engine.

BTW, I think Rafael makes the right response to Steve Rubel’s idea that “It’s very difficult to survive as a paid service in a Long Tail environment. One reason is that it’s now easier to discover free, open source alternatives.” Rafael replies that services like EngineeringVillage add “value to the content that we publish (indexing, writing abstracts), creating better searching features and providing analytical tools (intelligence).” The Long Tail enables the creation of such deep value that only some of that value can be addressed by Open Source solutions (long may they wave).

(Disclosure: Steve Rubel works for Edelman PR, to whom I consult, and I recently did some videoblogging for FastSearch.)

[Tags: faceted_classification search tagging folksonomy patents everything_is_miscellaneous long_tail ]

We can already tag videos, of course. But how about being able to tag the good parts?

YourView lets users mark segments on a video using a set of icons, and also indicate the "intensity" of each. In their example, a user could tag all the serves in a tennis match, and then watch all the high-intensity ones, or could watch all the non-boring parts of a cricket match, reducing a 44 hour match to 4 seconds and the credits. More to YourView's point, the broadcaster of the video could mark it up with icons.

This isn't exactly tagging because the user only has access to a pre-determined set of icons (and it's not clear from the site who determines the set). It's also not clear whether user-based markings are public and social; I'm assuming not. So, you don't get the social effects of tagging, e.g., find the segments of a video the most people have marked "great shot" or find all segments of all videos anyone has marked "whoops."

It requires the use of the YourView viewer. Enable any user to do this outside of the YourView viewer, and you'd really have something. (I'm not saying it'd be easy.)

MotionBox.com has a related function that lets you select any portion of a video and tag it—real tags—with any words you want. It seems that only the person who posts the video can tag selections, though. And you have to view the video on the MotionBox site.

Still, we're getting closer... [Tags: video motionbox yourview tagging folksonomy everything_is_miscellaneous ]

Siderean, a faceted classification company, has announced a patent for what it calls "relational navigation."

Faceted classification lets a user browse a field in typical hierarchical fashion—like navigating through the nested folders on your desktop—except the hierarchy is created dynamically as the user decides which property matters to her now. So, instead of having a fixed taxonomy that first divides all books into fiction and non-fiction, and then subdivides them by language and then by year, with a faceted classification, a user might decide first to find all the works written in the 19th century, then drill down to the non-fiction, etc. It has taxonomy's virtue of guiding navigation without its vice of having to present the user with one and only one path through the taxonomy.

Faceted classification and taxonomies both work by showing the user narrower and narrower results . That's often what we want, but in this crazy world, we may also want to leap off the branch we've walked onto. Siderean's relational nav shows context from branches outside of the path you've walked. Siderean refers to this as the ability to "pivot," as in a database pivot.

Techniques that let us play with the dialectic between narrowing our focus and expanding it—searching and discovering—are all to the good. The faceted classification industry overall is up to important and exciting stuff. [Tags: everything_is_miscellaneous taxonomy faceted_classification facets siderean ]