Monrai Blog

Razorbase Safari Bug

2009-06-07T16:23:00.000-07:00

UPDATE: The Safari bug is now fixed.

There is currently a bug in the xslt stylesheet for Razorbase which prevents Safari from displaying the main results page. No resolution is availbile yet, but I'm working on it. I apologize for this inconvenience to Mac/Safari users. I'll post once the bug is fixed.

Razorbase Examples Part 5

2009-06-06T00:15:00.001-07:00

If the litmus test for a killer app is the ability for users to create fun, ad-hoc games with it, then linked data is the Semantic Web's killer app. Here's an example of a variation of the Kevin Beacon game using razorbase.

If you discover any new games using razorbase, please post the rules as a comment.

Razorbase Examples Part 5

View more OpenOffice presentations from sdmonroe.

Razorbase Examples Part 4

2009-06-05T19:24:00.001-07:00

In these slides, I walkthrough the use the file system metaphor in the razorbase UI.

Razorbase Examples Part 4

View more OpenOffice presentations from sdmonroe.

Razorbase Examples Part 3.5

2009-06-05T14:45:00.001-07:00

I've added "type actions" which allow you to do multiple-step walks through the database with one click. Currently, actions have been added for types: Category and Person. More will follow soon.

As a side announcement, a re-crawl of the LOD is expected to be complete between today and mid-next week. This means less errors in the data, more current data, and less mal-formed characters.

Razorbase Examples Part 3.5

View more OpenOffice presentations from guest02493a3.

Razorbase Examples Part 3

2009-06-04T14:47:00.001-07:00

There is a ReadWriteWeb article about Google Squared, a new labs product that gives structured results to keyword search. It's Google's version of razorbase. The article expresses some of the services shortcomings using a search for dog breeds as an example. In response, I've created some slides that shows how a lookup for dog breeds works in razorbase. It's a great example of the universe of Linked Data vs. the closed world of Google's document web.

Razorbase Examples Part 3

View more OpenOffice presentations from sdmonroe.

Razorbase Examples Part 2

2009-06-02T20:31:00.001-07:00

I've released a second presentation on razorbase, this time showing the alternative ids action in action. I also give a demonstration of what isn't allowed to happen when people don't adhere to the Principles of Linked Data.

The world is refining how it consumes, people are demanding goods and services to be delivered in ever decreasing sizes that allow the consumer tighter control and choice over exactly what is consumed and how much of it. Cell phone billing went from long-term service contracts that spanned years, to pay-as-you-go plans and the concept of more grainular roll-over minutes gained popularity. Amazon has figured out how to break the whopping utility bills of their server centers into small, per-usage units which the consumer pays for as they go. Youtube turned the world of media on its head by allowing users to instantly share bit-sized video clips that can be consumed faster than it takes to clean your email inbox. The notion of buying a record album has been usurped by a iTunes feature that allows for purchases at the level of a single track of the user's choice. One could refer to this trend as the atomization of goods and services, and it's about to happen in the realm of the WWW (and data in general), and in a very big way.

I've created and used many Linked data/Semantic Web demos over the last decade, and each has taken a level of... imagination, in order to feel where this is going. Razorbase is the first time I've been able to really sit back and see our destination physically on my computer screen. The single most revolutionary feature of this service is the Information About page, which lists all properties of the subject in focus. If the subject contains a list of things, then you'll see all the properties belonging to all the results. Most linked data browsers I've seen attempt to push the entire description of a URI resource into the UI at once (or at least as much as one results page can hold). This is a very pre-atomization approach, akin to how a Wikipedia article is displayed. If you look up WWII on Wikipedia, you could easily spend an hour scanning the article and taking notes. The user would benefit more by having this information sliced and diced into bit-sized, labeled chucks, and placed on a platter for them to choose freely from... in a word, atomized. As long as the user has a general notion of what properties are (probably) availible, and what they can expect to be on the other end of those properties, then the user is able to (in most cases) easily build a mental map or strategy of how to get to the exact information they're interested in, no matter how complex or esoteric the connections involved, no matter how heterogenous the sets of data are standing between the user's start and end point. Once the user has a mental map, they only need 1) paved roads and 2) the right sign posts to keep them on their track. Razorbase provides sign posts by displaying blue arrows that take the user back and forth through the dataspace. It provides paved roads by e.g. using collaborative-filtering-esque techniques for exposing the most worn paths through the dataspace.

Razorbase Examples Part 2

View more OpenOffice presentations from sdmonroe.

Razorbase vs. Parallax

2009-05-30T18:19:00.001-07:00

For those who don't know, David Huynh, most noted for his work at the Similie Project, released a faceted browser for Freebase (now part of the LOD dataset) earlier this year. Much of my work on facets/set-based browsing is based on his. Here's a compare/contrast (see presentation below) of his Parallax browser with razorbase that may be useful.

There are now two new buttons/actions that came out of earlier observations of how people interact with the razorbase UI: mutual connections and descendants.

Mutual Connections: allows you to view the mutual connection of a certain type linked to the subject, with one click. E.g., if you were viewing Someone's friends, this button will take you to their friends.

Descendants: allows you to view the descendants of a connection. If you were viewing People who influenced someone, this button will take you to people who they influenced. Use to pull friend of a friend, a person's ancestry, etc.

Initial experiences:
I've found that what folks discover is most useful about the service is the ability to develop their own strategy for finding the information they need. By refining my criteria through trial and error browsing, I was able to find valuable Web resources about an esoteric research topic: "Recommendation Engines/technology". The results were several orders of magnitude more precise than Google and Wikipedia yielded.

I began my looking for things named Recommendation Engines, which returned all things with that title in its name. I then drilled into the Documents and Articles category. From there, I examined each article in the list, and manually collected a list of companies that I found to interest me. After pulling a list of companies that develop said technology, I was able to go to the Websites that published stories about those companies, e.g. by looking up Companies named riffs.com. Using the mutual connections button, I found the other stories published by those sites (because in most cases, the site containing the story were sites dealing specifically about my topic). From there, I figured that those stories probably contain links to stories/companies/web services related to my topic, so using the Information Explorer, I pulled all links referenced by those documents, and got a great list that yielded more companies in that space. Then I figured, the links in those documents may also be related. The descendant button allowed me to pull the links two-degrees out from the original list, yielding a list that was less relevant, but which did contain a few precious nuggets. In the poverty of data regarding my esoteric topic, the ability to locate those few nuggets by 1) defining a criteria based on Category and other information about my topic, and 2) drilling through and cutting the results, delivered value that I truly can not find anywhere else on the Web. This was my first real-world experience with the value proposition of the linked data web. The resulting presentation was one which I would not have been able to compile otherwise within the time constraints I was given.

Razorbase vs. Parallax

View more OpenOffice presentations from sdmonroe.

How to use Razorbase

2009-05-19T10:33:00.000-07:00

Razorbase is a browser for discovering and exploring connections between things (people, places, movies, shoes, food, etc). It does this by querying not the World Wide Web (a global network of websites), but the burgeoning Linked Data Web (a global network of databases). Try it for yourself and see whether you can discover the difference between the two.

This is a tutorial for the service. My goal is that the browser be so intuitive, that you could beam a caveman right in front of it, and he could figure out what it's for and how to use it without being told. Well, so much for that :) (Update: Slides are now available here)

Homepage: There are two controls there of interest, named link, and the query text field. Click the named link modifies the type of query to perform, options:

things named... (e.g. things with "Bill Clinton" in its name or title or label)
things connected to... (e.g. things connected to fencing)
things known by the URI http://... (i.e. things known by the URI http://www.google.com)

Filters: Razorbase allows you to define complex filters to restrict the items in your results (see Navigation and Zooming below). Click the Your query link to view all filters. The Your query section contains a breadcrumb list/trail so that you never get lost while browsing. Click any node that appears there to go to that subject. Access all nodes by summoning the filters (click 'Your query'). The last breadcrumb in the trail is what you are viewing, it's called the subject.

Group results: You can view the categories for results by clicking the Category Explorer icon (magnify glass), if you want to see all results ungrouped, click the Back to Results icon (blue left arrow)

All info about something: If you want to know what all information is available about the items in the results, click the Information Explorer icon (blue circle with exclamation mark). There is a other info link that appears on the blue bar, click that if you want to see further types of information, then click main info link to go back to main information about the results.

Navigation: You navigate through the dataspace by clicking the blue right arrows, clicking one will take you into whatever it's marking (so Friends >> takes you to the friends of the subject).

Zooming: Zoom in and out of categories by clicking the magnifying glasses next to category names under the Category Explorer icon.

Add filter: The blue plus sign creates a filter on your search by binding the item as the value of a connection (e.g. all people whose email is emailToFilterBy@somesite.com). Be aware that Navigating and Zooming also add filters (but the values in the case of Navigation are unbound). So basically, the blue plus sign does what the blue left arrow does, but instead of being taken to that item, you're taken back to the subject you just left.

No text please: Don't want to have any text search in your query? For example, your viewing Presidents connected to Marilyn Monroe, and you want to drop the criteria that they be connected to her, resulting in all Presidents. To do this, click the Your query link to summon the filters, then next to the text field, click the blue minus sign. This sets your text to anything. Click the blue plus sign to add some text.

Little or no results? Razorbase's novelty is enabled by the fact that OpenLink has figured out how to give SPARQL query access to a quad store of over 4.5 billion triples. In layman's terms, this means we have to ration out how much time the server can use to perform your query. The default is 2 seconds (i.e. the server gets all results it can find in 2 seconds or less). So increasing the time can potentially increase the number of results you get. To increase the time, click the clock button when it appears. Each time you click, the time increases by 2 seconds (up to 12 seconds for now).

Side notes: The power of this UI approach is two-fold, faceted-browsing, which allows you to navigate large set sets by filtering data as you go. The second is set-based browsing, which allows you to see information about multiple results simultaneously.

Hope that helps :) Next, some strategies I've found in my interactions with razorbase.

Discover Connections between People, Places and other things

2009-05-18T14:12:00.001-07:00

I present a new linked data browser called razorbase, for discovering and browsing connections between things. In the next few days and weeks, I'll be blogging guides that introduce some of the features of the browser, as well as some helpful hints I've discovered while interacting with it...

In the meantime, next time someone asks "Where can I see the Semantic Web?", you can finally reference something touchable :)

Welcome to the Game of the Decade!

2009-05-13T07:05:00.000-07:00

A second provider of traditional search (see first) has now entered the Semantic Web, or at least has dipped their toe into the pool. This is encouraging news for the Semweb, it validates the merits of structured data. However, traditional search engines have historically been apprehensive about structured data, and can you blame them? After all, they're in an industry built on a major deficiency of the WWW. Most data you see on a web page comes from a structured database. As an entry in a database, there is no mistake about the info connected to a Review, e.g. who wrote the review, what the subject of the review is, or whether the review is positive or negative. But back at the genesis of the WWW, document retrieval and sharing via HTTP is the only thing that had been... worked out. And the WWW grew so big so fast, that they didn't have time to do it right. People wrote programs that queried the databases, and exported the results into documents written in a standard created by Sir Tim Berners-Lee so that your computer knew how to display it on the screen (e.g. make title large, place these reviews in a table with author and score in each row). But this step destroyed the structured expressed by the rows and columns of the database. The meaning of the data in the webpage had been lost. So search engine set out to divine the web page for structure post-mortem, and some approaches worked out better than others.

So, the WWW is broken, and as long as it's broken, traditional search engines have a place in the market. If the web had been connected at the level of the database to begin with (instead of at the document/webpage level), then web page indexing methods would have absolutely no value today. If the WWW is ever fixed, then traditional search engines may have a non-trivial problem to face. When the W3C announced a new recommendation to create a World Wide Web of Databases, this gave the world a push into the direction fixing the WWW fundamental problem. As we inch closer to that reality, the emphasis on search will diminish and ultimately be replaced by the notion of lookup, which is a new game, and which brings fresh, new opportunities for both consumers and entrepreneurs.

More Serendipity

2008-10-22T10:19:00.000-07:00

[Warning: the links in this post are not static, and may not exisit for long]

The online Cypher demo has some simple demo lexemes for the dbpedia, such as relationships between writers, books, films, actors, among a few others. I ran some very simple inputs, and serenpitiously noticed some interesting facts (new knowledge for me):

- I always thought Angelina Jolie and Brad Pitt were married, and also, I assumed that Billy Bob Thorton was her only previous husband, but Cypher says not so. By sending a plain and natural statement describing my interest, I was able to instantly discover knowledge I thought I had (based on dbpedia):

the husbands of Angelina Jolie
Billy_Bob_Thornton
Jonny_Lee_Miller

- Cypher said that only three US Presidents have ever written books, I would have thought every US President since, say 1940, would have written at least one book (click the book link to see the author and other metadata for that book):

the presidents wrote what
An_American_Life
Palestine_Peace_Not_Apartheid
The_Reagan_Diaries
Commentary_on_Palestine_Peace_Not_Apartheid
Peace_Is_Possible_%28book%29
Giving:_How_Each_of_Us_Can_Change_the_World
My_Life_%28Bill_Clinton_autobiography%29

- Cypher says that only three First Ladies have every written books, I would have expected more

the wives of presidents wrote what
An_Invitation_to_the_White_House:_At_Home_with_History
Living_History
It_Takes_a_Village
My_Turn_%28memoir%29

- There have been more US Presidents who have been in/on film than have written books (could Andrew Johnson have possibliy been in a film???, or is that an error in dbpedia?)

the president who acted in a movie
Andrew_Johnson
William_McKinley
Andrew_Jackson
Richard_Nixon
Ronald_Reagan
Jimmy_Carter
Theodore_Roosevelt
George_W._Bush
John_F._Kennedy
George_H._W._Bush
Dwight_D._Eisenhower
Bill_Clinton

This type if serendipity is only possible once the barriers to such queries are lowered by tools like the Cypher. Cypher is a key part of the infrastructure for the semantic applications of tomorrow, where data and functionality is pulled down to the user on demand, through tools like ubiquity firefox (http://tinyurl.com/5m9lhb) , as opposed to the pre-packaged market of today where data is delivered as web pages. In this vision, the results of SPARQL queries, articulated as natural language descriptions and statements, will be feed directly into other services to produce dynamic, complex functionality, and deliver services to the user that the data and service publishers could not intend.

Web 3.0 Conference

2008-10-14T18:44:00.000-07:00

I'll be attending the Web 3.0 conference this week in Santa Clara, CA. I'll be moderating B8 Session: Approaches to deep customer relationships, and I'll be speaking in T2: It's a Short Way to Tipperrary The Role of Schemas in building smart apps. Also, those who plan to attend can catch a demo of the upcoming release of Cypher in the Monrai booth in the exhibition area!

I plan to do some blogging from the event, so stay tuned over the next couple of days.

How to Insert Web Links Using Ubiquity and the Summon Command

2008-09-20T14:56:00.000-07:00

I recently complained about the tedious labor involved in linking to stuff, such as the links within this article. So I created a ubiquity firefox (http://tinyurl.com/5m9lhb) command that allow you to get the link representing any text, and insert it into a web page. Install the command here. For example, if I wanted to point to the show Firefly (http://tinyurl.com/6g3zfj) , I only do one step:

1. Call ubiquity firefox (http://tinyurl.com/5m9lhb) using the ctl+space command (ctrl-alt for mac), then type summon text.

And that's it. When you press enter, the link for text is automatically inserted into the page you're viewing. You get a list of about 10 choices, choice #1 is inserted by default. If you want to choose another from the list, just type pick number after your input: e.g. summon bill clinton pick 3 will insert the 3rd link from the list of possible links to pages that represent Bill Clinton. The first item in the list is always the page returned from Google's I'm Feeling Lucky (http://tinyurl.com/6qs8fq) search.

This command is great for quickly inserting links into email (http://tinyurl.com/692yha) , blogs (http://tinyurl.com/56u7ys) , blog comments, etc. Optionally, you can just select some text in the page, and ubqi summon this to replace that text with the link.

It uses ubiquity firefox, sindice (http://tinyurl.com/4fabtc) to do the URI lookup, OpenLink (http://tinyurl.com/4dp9s2) data explorer to view the URI description, and Google (http://tinyurl.com/3gsc6g) for Feeling Lucky results, and delicious (http://tinyurl.com/4jguko) for bookmark search.

Nice to haves:
- search your personal dataspaces for links (delicious, etc)
- conditionally insert only title link (for Rich text editors like gmail) or title and parenthetical URL (for non-rich text editors like twitter)

Update: here is another insert link command, pretty cool stuff

Semantic Web Value Proposition

2008-09-11T11:46:00.000-07:00

Dan Grigorovici has been blogging a lot lately about nailing down the Web 3.0 value proposition. I think that we as Semantic Web evangelists must also be good salesmen, and therefore need a good pitch. I think two of the big value propositions for Web 3.0 are: Automate Tedious Tasks and Seridipity/Knowledge.

Automation
The automation value prop came to mind as I experienced this real-world use case recently:

My nephew needed a video of Julius Cesear for his class, so I thought cool, let rent it. We then did the following:

Collected the numbers of video stores in the city (via hard copy of yellowpages)
Called each video stores to check availibility
Hold the line as the clerk manually searched on their inventory
Mapquest each store and determine the closest

My grandma's personal database automates this tedious process. It allows you to summon stores near me that have movies about Julius Cesear and get a result set of all stores, and the "proof" of why that store is in the results, i.e. the properties involved in the query and their values, including location, the movie, movie's description, and any other property that caused it to appear in the results.

Serendipity
The serendipity value prop is also important articulate. I hear the terms data, information, and knowledge listed a lot, accompanied by someone's definition, so I will offer mine to help frame this point:

Data = a set of things (e.g. a list of shoes, a table of dates)
Information = statements about data (e.g. the price of this shoe is $60, your appointment is on this date)
Knowledge = statements about information (e.g. your appointment happens to be on the 3rd anniversary of the day you purchased this shoe)

Thus, the ability to have knowledge represented in the Semantic Web leads to the idea of serendipity, which (if I may borrow a term from information retrieval) helps us to increase the recall in our lives, the missed opportunities and overlooked connections that can improve the quality of life and cause of to be much more productive. Serendipity is a concept that a hear tossed around, and so I will offer a concrete example of what this means. I was running some test input through Cypher, and summoned the authors who starred in films. I was surprised to see Adolf Hitler at the top of the list (the list was alphebetized). I naturally assumed this had to be a bug in the software, so I looked at the full results page, which included the proof. And there, I found that Adolf Hitler indeed was in several German propaganda films. That notion of learning that something I once thought was unlikely was actually quite likely, that's serendipity.

Virtuoso + Cypher = Dream Team, or Check Your Foundation!

2008-09-11T10:26:00.000-07:00

I'm not an easy customer when it comes to technology selection. I've come across a lot of seemingly interesting frameworks, etc, that had a great vision, but were just done wrong, and not having a piece is a lot better than having a piece that's done wrong.

The Semantic Web is one of those things that can easily be done wrong if you're not careful. A good example is how people were adding data to the semweb before the insurgence of Linked Data. At that time, people were just putting Turtle and N3 in .zip files and posting them to the web, we didn't know any better. Time and experience has lead to better design philosophies. And one of the very few companies I've encountered in this space that is doing the Semantic Web the right way is OpenLink. I was cynical when I first was exposed to some of their products, like the data browser, and the query builder, because it's not easy to seperate those who execute right from those who don't. Let me deviate a second here and give you two examples, SIMILE and Sesame (these aren't negative reviews or anything, as both of these projects are profoundly innovative, but just some short comings I've experienced which may be improved in future versions).

SIMILE
These guys have some really powerful concepts oozing from their labs, their Longwell/PiggyBank browser was the first true RDF browser, and it was the first time I saw anyone implement faceted browsing. When I heard that they released Exhibit to take these core concepts and streamline the framework for quick/easy development, I immediately jumped on it and used it as a result set browser for Cypher (which at the time lacked a UI for browsing result sets). They had the concepts down, but the implementation turned out to be wrong, and if not corrected, will cause Exhibit to be unusable in any real Semantic Web app. These include:

JSON input: Exhibit takes JSON data and uses it to generate the browser. This is great, since lots of Web 2.0 apps speak in JSON. But it's cumbersome when Semantic Web apps want to display results of SPARQL queries (varible/value pairs called tuples). And since Sesame has a built in JSON formatter for result sets, I only needed to extend it to suite my need. Good, right? Well, in order to produce a JSON output that had the required properties for Exhibit (i.e. label, id, etc), plus the 'evidence' properties (i.e. the properties which causes a resource to be included in the result set), I had to generate a SPARQL query containing about a dozen variables, some of which had multiple values for each result, in which case you encounter the permutation problem, which can lead to tens of thousands of rows representing only a few actual resources. As the response time progressively increased, I needed a solution quick. Also, Exhibit didn't like the fact that I was sending URIs as property names, and had a problem displaying them in the interface. The JSON tweaking took about 24 precious hours to complete, code that eventually was tossed anyway.
No sense of linked data: Beacuse Exhibit wouldn't recognize the URIs I sent it as Web assessible URLs, I was not alble to simply click a link and get a new results browser.

Openlink Data Explorer
In order to attempt to remedy the Exhibit problems, I asked Kingsley if there was a way we could solve the permutation issue with tuples, perhaps in the Virtuoso query processor itself. He then gave me a huge epiphany, SPARQL Construct queries. Construct was one of the areas of SPARQL that I understood, but probably had never actually used because I didn't really see the value in being able to make a graph from a query, since you had the tuples resulting from a Select query, and could just construct your own graph from those results. But Kingsley pointed me to a page which took as an argument a SPARQL construct query, then allowed me to browse the resulting graph. No change was needed to Cypher, except to replace the Select query with the Construct query (i.e. add a construct clause to the query whose paths are basically the select paths), and BOOM!, I was browsing my Cypher results set!! The entire integration took about 30 mins to implement and test, only because I had to make the change in Cyparkler because I (encouraged by this) decided to go ahead and enhance it to support Describe and Ask as well. The ease of point-and-play integration is what the Web 3.0 is all about.

Sesame
The Semantic Web community has not yet agreed on a standard synonymous with JDBC or ODBC, i.e. a standard for making transactions against and managing triple/quad stores in a platform agnostic way, and it makes for integration between frameworks difficult. So your framework winds up getting "wedded" to one platform or the other. Cypher and Sesame have been married now for about 7 years. It is not only a triple store, but more importantly, it is a framework for making transactions with and manipulating triples stores in Java. Jena is another. Sesame's API called Storage and Inference Layer (SAIL) allows for plugging into other types of back-end triples stores.

Problem: Virtuoso is a great product, but was notorious for requiring you to either write straight SQL or SPARQL, or use stored procedures or use the proprietary VSP language to interact with it. I.e., there was no API or API implementation availible to help bootstrap my Virtuoso integration.

Solution: We simply wrote a Sesame Repository implementation which: 1) connects to an instance of Virtuoso via JDBC, and 2) connects to any SPARQLEndpoint via the SPARQLEndpoint protocol. Now I get all the benifits of Virtuoso while leveraging all the framework code I have for Sesame.

Result: Once upon a time, Cypher could only store data to and query Sesame HTTPRepositories out of the box, but the problem is that Cypher greedy about data, the more repositories it can connect to the better it's performance. The number of publicily availible Sesame HTTPRepositories serving data live on the web can be counted on one hand, so the demos were... well, boring. But now, Cypher can now attach to any one of the growing number of SPARQLEndpoints or Virtuoso databases coming online, right out of the box by configuring a connection in the startup properties file. This was a huge development, because overnight, I went from asking things like who knows Sherman Monroe (from my FOAF flat file) and Sheman's address (from my FOAF flat file) to fun stuff like the presidents who were influenced by people who played in films (from dbpedia).

Semantic Web + Cypher + Ubiquity = My Grandma's Personal Database

2008-09-11T09:07:00.000-07:00

The idea for Cypher came to me a few years ago. The vision was vivid and complete, replace my mouse and keyboard for a headset (mic and speakers), allow me to talk to my computer to issue commands and summon "things". I remember drawing an interface for a web browser that had no buttons, and no menu bars, just the content of the web page. Saying a link would click it. I called this embodiment "Lewy", why for I know not, but which later became 'LUI' for Language Understanding Interface, or Linguisitic User Interface. I was young and naive, and abruptly took a sabbatical from college without any idea of what was required to make this real, but one thing I was certain of... if I could imagine it then it's complete possible. The resulting technology and it's industry has since grown by leaps and bounds, and when I was turned on to Ubiquity, I saw the final piece of this vision begining to be set into place. So let me talk about the first two pieces a little.

A Human Language Processor
The first requirment for LUI is a Human Language Processor. In my initial research, a great book called Symbolic Species made it clear to me that there are no short-cuts in NLP, if it's a "simple NL processor" then it's not really an NL processosr, because by definition, Natural Langauge is highly complex. This basically meant that I would need to figure out what processes are taking place in the brain while you're reading the New York Times. The task of NLP is a task in cryptology, thus the name Cypher. After 8 years, we finally have a framework for processing sentences like humans do. This is a 'cry wolf' type of statement, because of the many past promises and ensuing failed attempts of people/companies/instiutions of learning in this space. That's why I don't blog so much, instead, I'd rather spend that time setting up demos and releasing code, then let the work speak for itself, in every sense of that term :) (ok, ok, I'll stop :) So that part done.

A Universal Database
The Jetsons was a huge influence on me as a child. One of George's friends was an AI called RUDI (Referiential Universal Data Index). RUDI seemed to know everything, the entire body of all human knowledge. The WWW is the closest embodiment of RUDI we have today, with Google being the main interface. Cypher (the first piece) is dependent on a subset of human knowledge, called Lingusitic knowledge (i.e. a RUDI for language). The types of questions Cypher would pose to this database are: what is the structure of a noun phrase, does the verb 'marry' take a direct object? A preposition? How does one make the word 'ox' plural... all those language conventions you learned in elementary school. The WWW contains this data, but there is a problem. All the data is in human readable form, and would require an AI to extract it, which puts Cypher in a catch-22 problem. The solution, put all the required linguistic knowledge in a structured database. Done. The next problem is critical mass, most people don't realize this, but the amount of data a 3-year-old child has about language is astronimical!! It's nothing short of a mirical that childern are able to acquire language. The number of rules for combinations, phrase grammar, lexical restrictions, etc are innumerable, a certain critical mass is required for Cypher to work in unrestricted text. Since we don't already have an AI to populate this linguistic database automatically, we will need to someone do it manually. The Wikipedia has shown that a 'crowd-sourcing' approach is viable for this task. The Semantic Web allows a way to facilitate crowd-sourcing on a very large scale. The Semantic Web has been built, and the MetaLanguage Ontology (MLO) is now in it's first official release. So that part done.

Vision Without Action is Dead
So now my computer can accept my plain langauge phrase, ask the Semantic Web for a strategy for processing it, submit a 'coded' version of the post-processed input to the Semantic Web, then get a response (either a set of statements in Semantic Web langauage, or a set of results using Semantic Web URLs). That alone is really fun, and even in playing with the demo, I was able to find some very interesting facts. But remember, the vision has two parts, summon things (i.e. the nouns... done), and the second part was executing my commands automatically, i.e. the verbs, which takes us back to the beginning of this article... enter Ubiquity.

Let's start with a practical example, at the beginnig of this section, I wanted to reference Terrance Decon's book. To do so, I had to:

open a new tab, google "Symbolic Species"
click the link (because Google's result set links are googlfied)
copy the URL from address bar
take a breath
nav back to this tab
select the text for the link
paste the URL
repeat for all other links in article

... and this is 2008! The vision is to be able to summon a resource by saying Symbolic Species or Terrence Deacon's book and have my computer return the Amazon link (or whatever the net-citizens agree is the URL that represents that resource). This idea of 'summon a resource by description' is the piece of the vision that Ubiquity addressed. I wrote a prototype (just follow Ubquity instructions for installing it) which takes a natural langauge phrase (e.g. Terrance Deacon's book), and returns a table containing the list of 'answers'. It's only a "sound check" prototype, it only queries dbpedia for now. The vision for the plugin is to be able to select some text in a page, call a ubiq command like "get this", or alternatively type get Terrance Deacon's book, then have Ubiquity insert the link into the page or editor. The plugin will allow you to do this for anything that can be described, so summoning from your personal dataspace + the global dataspace, things like: my sister's boyfriend's alma mater or my grandmother's birthday or my car's gas milage and in response it inserts the link or text representing that thing.

So that part... in progress....

Cypher 1.9 now ready

2008-09-11T09:03:00.000-07:00

The long awaited Cypher 1.9 release is now ready for deployment, and is running live in the wild. The release will be deployed over the next couple of days, we thanks everyone for their patience with this release, and the many push-backs due to a huge overhaul of the framework.

I will be blogging about some of the new features that you can expect from this release, how this release is related to other interesting Semantic Web projects, and what is on the horizon for Cypher. Brb!!

Case for NLP in Semantic Web

2008-07-28T09:09:00.000-07:00

Here is a great paper on the case for the use of NLP in Semantic Web applications.

CNL_Reportv7.pdf (application/pdf Object)

Response to Dan Grigorovici's SemanticWeb.com articles (Part 2)

2008-07-08T14:20:00.000-07:00

I finally got around to looking at MicroFo...s today (which I will henceforth refer to as MFs, if you know what they are, keep reading, if you don't know what they are, good!), and was amazed at how redundant it is. It's primarily RDFa watered down. But then something else struck me... it is actually being USED, by a growing number of software developers, web publishers, and end users. It seemed to catch fire rather quickly (relative to the growth and development of the W3C endorsed Semantic Web).

As I continue offer my response to Dan Grigorovici's blogs on the business and marketing aspect of the Semantic Web (or more accurately, the lack thereof from his prospective), I would like to turn the attention to the growing attraction to MFs over RDFa as offering one of the sources of the problem. According to the Wikipedia page, MFs is a grassroots effort, which gained popularity and then support from a corporate sponsor. It's not a standard, and is steered by the very loose-knit community, essentially, a mf begins it's life as a wiki entry. Question: What drives an end-user's decision to "roll his own" verses adopting an off-the-shelf solution? Answer: Familiarity. Neurologist know that the task of creating a new synaptic connection in the brain requires many times more resources and energy than reusing an existing neural pathway. So reuse, leveraging existing knowledge/practices is an aspect that is always sought after by the brain. It's often times infinitely easier for a developer who is proficient in HTML/CSS, to build a site from his own personal libraries of (familiar) templates, than to be given an existing site to remodel. And with MFs, and the Web community at large, it appears that there is less friction involved in starting from HTML --> HTML+semantics, and easily seeing the relation, than to one day have a RDF Primer and SPARQL specification dumped in your lap and told to go from QuadStore --> triples --> HTML+semantics. It's obvious that the Semantic Web vision is "right", because the world is obviously demanding it, but the world never wants to DO right. We must ensure that we arrive at the right destination, and get there the right way. If I set out to the store, and drop all my money on the way there, then what good is it if I make it to my destination?

I think that the momentum building around MFs indicates that the W3C's Semantic Web problem is largely one of PR and public image. If you are a good spin-doctor, I recommend the Semantic Web as a qualified client. I would go so far as to agree with Dan's arguments and Hank William's suggestion, that if you are a semantic web startup, drop the association with Semantic Web in any public or investor-facing collateral for your product or company. I really wish someone would have taken the RDFa spec, dropped the RDFa label, slapped the name MFs on it, stood up a wiki and discussion group, and proposed it from the grassroots level.

Keep in mind that by rebranding the Semantic Web, we mean only a cosmetic overhaul while leaving untouched the core the tenants we have, as a community, chosen as necessary to uphold: use dereferencable URIs, publish, use and reuse RDF vocabularies, reuse other's URIs for naming things when possible, among others. It's important that we hold true to these principles, and that we are not blown about by every wind of doctrine that surfaces regarding how the Semantic Web should manifest, or compromise the vision we've laid out. I, for one, have decided not to support MFs in any of my software, nor publish it on any of my sites. As a community, we have our view, and the rest of the world seems to have their view, let us stick to our view, and not give way to these factions. Many of us are involved in work to RDFize the existing world, as Kingsley Idehen puts it, to "build the house around the users", as opposed to dragging the user into the house. This is strategic and a form of compromise, but it can only go so far. The Semantic Web will need solidarity if it will be preserved, multiple standards split resources, focus and impede interoperability and reuse, and these splits will only weaken our foundations. Only by having one unified set of standards, emanating from one unified community, can we hope to build anything substantial on top of the Semantic Web.

Sources:
Why does everything suck?: Killing Ontologies/OWL In The Semantic Web?

Response to Dan Grigorovici's SemanticWeb.com articles (Part 1)

2008-06-21T20:48:00.000-07:00

Dan Grigorovici, AOL exec, semantic web evangelist, and good friend of mine, is doing a series of articles for SemanticWeb.com. In it, he makes a call to action to the semantic web community, and admonishes us to buckle down on the PR aspect of the semantic web (or the lack thereof). I'd like to attempt to offer some responses to the issues he addresses.

I was having a conversation with a few of the "usual suspects" of the Semantic Web evangelical crowd, and it was mentioned that one of the problems we face is how do you make money in a medium that is not eyeballs-driven. Because the semantic web is a technology who users are mostly made up of machines, the catch-all monetization strategy of Web 1.0, advertising, does not apply. Someone then took the words out of my mouth by saying something to the effect that you don't have to have people looking at a web page to deliver an ad to them, the ad can be delivered across other mediums, SMS, etc. The problem is that current advertising is obtrusive. I am reminded of how one time I had a really big headache, and I went to the drug store to buy aspirin, and found myself in the aisle asking "Now, what's the name of that 'I have a headache this big' medicine?". A classic example of a good product that was offered to me at a bad time. So an improvement that is needed is the injection of "context" into the equation, being able to deliver to a user a product, service or opportunity, at the most opportune and relevant moment, based on their current need, time, and place. At the time I saw that commercial (btw, the brand is Excedrin), I was a small child and probably had never had a headache. But when I finally entered the market for it, I was unable to find/recall it.

I added that user behavior, interests, etc can be collected (by permission) and used to drive more intelligent referrals for purchase decisions. People are always looking for better advice before buying, case in point is an experience I had with a crummy airline. Had I had a service that could have made a quality recommendation on my airline (taking price, quality preference, and other factors together), I would have literally saved hundreds of dollars.

So one business model that will, IMHO, be a bread-and-butter source of revenue for Semantic Web companies will be in "cooking" triples that describe users and the things that interest them, to provide the knowledge/intelligence needed to fuel the next generation of recommendation services. These services will connect customers to the things they want and need with laser-like precision, and will deliver these laser-beam recommendations unobtrusively across a myrid of channels. Companies will pay a premium to have their products and services delivered by such services.

Stay tuned for the next micro-post, where I continue to offer monetization suggestions for Semantic Web startups.

The High Definition Web

2008-06-19T17:54:00.000-07:00

A lot of people have requested the presentation "Emergent Data and Semantics From Social Collaboration", prepared by Soren Auer and myself, for the Linked Data Planet 2008 Spring conference, so I have placed it online. (For now, as you read, please refer to the slides, I'll get some images posted soon).

In it, I expound on the trend towards a more High Resolution Web, or High Definition Web, where machines are able to see a richer description of people, places and things. Here is a bit of my notes from that talk for those who could not attend.

When we talk about Social Collaboration, as social creatures, sharing is a natural component of our evolutionary adaptation. The internet provided an infrastructure to connect computers, and WWW provides the means of performing this inherit human behavior of sharing, mainly documents across that connection. One of the greatest contributions the WWW made was that I could open a text editor, write something, then instantly share it asynchronously with someone across the world. But the document web limits sharing, i.e. the Social part of the WWW. Here are a couple of analogies that help illustrate this notion:

The Teller that Couldn't Tell:
- Suppose you deposit money, then I request a withdrawal, and the following conversation ensues:

You: I'd like to withdraw $20.oo please
Teller: Let me search for that, I’ll be right back... Ok, I found $10 that may be yours [or] I found $5 of the $20 you requested. Instead of telling me the amount you deposited, can you tell me what you had on when you deposited it, that may help me cross reference and find your deposit better?

Ridiculous, right? What's hindering the teller from delivering exactly what you requested?
-No matter if it’s an entry posted to my blog, or link sent to your email, or a dissertation in a PDF, or a web page, the only reason we have a notion of “search” and “results found”, is that documents are inadequate data containers that wind up suppressing the information we intend to share. The WWW, email, blogs, delicious bookmarks, etc., the document always looses important parts of the data we place in it. Because of this, the document must be searched for and founded again.

The Powerless Boss:
Suppose you have a boss who has a collection of many thousands of photos stored on your PC. He asks you one day to find a certain photo he took at a conference, he describes the photos in vivid detail. The problem is, you have this incredibly low resolution monitor, the figures in the photos are blurred beyond recognition, you can’t make out any of the people’s faces, how on earth will I you find the photo he's interested in? So you begin creating alternative heuristics for finding the photo, you think "he said he took it along side three people, there are a few with four human shaped objects, I can try to determine which one is him by cross referencing and narrow down…, well, he also took one that day at the podium, thankfully there’s only one with a human shaped form at a podium looking thing… and it’s shaped like and is the same color as the blob in this photo… one of these three are most likely him." So you email him the candidates, he prints them and selects the correct one, then says “Thanks so much, now I need the photo of me discussing the market data powerpoint slide”. Based on his feedback, you make a note that says “The tall purple blob in these photos is the Boss”. But then you then explain to him, "Hold on boss, all the detail you provide in your request is useless to me" (then you explain to him the situation)... "you’ll have to speak in terms of colors and blobs (i.e. please dumb down your request)".

He says: “Hmm, ok, the picture I want should have a tall, slender, dark blob left of center, and three smaller blobs to the right, because by that time two of the panelists had not gotten there yet”. Two photos match, you send, boss prints and selects the correct one from what you gave him, and you use that good guess to improve the heuristics in your little book. Your monitor’s terrible resolution introduces a tremendous pain for your boss, but gives you great job security, because of the tremendous value your book of heuristics now offers.

But now, let's take a look at what happens the moment your boss increases the resolution of your monitor:

Your book of heuristics becomes worthless
Your boss can now fire you anytime and hire anyone else to retrieve his photos
Most importantly, your boss can now request a photo from 1000s by describing the photo he wants in vivid detail, and can be fairly certain that he will receive the photo he request (if the photo exists), so he can say things like "I need some photos for my homepage, get me all photos of me taken when I still had a beard, and taken outdoors wearing no suit, at my home, or taken at a bar with anyone I know"

Now think of the description of a resource as a photo of that resource, and each statement (triple) involving that resource as a pixel that makes up the photo. Because documents were the atomic unit of information, the web had a really, really, really low resolution, and Google held a very valuable book of heuristics. As we increase the resolution of the web, the emphasis on "search" will evaporate.

The Trend Towards a Web in HD

What we’re moving towards with the Linked Data Movement, and the Semantic Web movement at large, is what can be described as a High Definition Web (i.e. Web 3.0, where each version increment roughly corresponds to a decade). The web has always been about describing things.

Web 1.0 contained statements where documents referred to nouns and you only had one verb isSomehowRelatedTo. Anchor tag is a reference to the relationship isSomehowRelatedTo. If you think of information (i.e. a statement) as a pixel, Web 1.0, if a document only contained one hyperlink, the pixels that make up it’s photo were few, or it may have many inbound and outbound links, but because each link means the same thing, it had no color (i.e. the link had no distinction)

Web 2.0 introduced subjects of several new nouns types, same monolithic verb isSomhowRelatedTo, and an object of type ambiguous term i.e. tag. Web 2.0 increased the number of pixels just slightly, but still no real color.

Web HD completes this transition by offering all named entities as subjects and direct objects, and any relationship as verb. Web HD is like having a life-like photograph of a thing, we can say this is a person, we can describe their phenotype, their genotype, likes, dislikes, social relationships…, each statement can now offer distinctly different information (so you have this wide range of color), and because you have this rich and inexhaustive vocabulary, the number of pixels in the photograph explode.

Linked Data Example

2008-06-17T07:54:00.000-07:00

I'm blogging live from the LDP conference, and have seen some very exciting technologies and heard some excellent presentations of the linked data vision. In my talk on tomorrow, I discuss the differences between todays web (Web 1.0 & 2.0), which is primarily a web of opaque documents and the simple "isRelatedTo" links between them, verses tomorrow's web vision which offers links between granular semantic (i.e. non-ambiguous references to self-described) concepts. Thus, instead of the document (and links between them) being the atomic unit of information, the database becomes the container.

But Kingsley today demoed something I had not thought a lot about... what if you make the document the container of these richer, semantic statements. RDFa is a standard for embedding RDF into HTML documents. But take a look at Kingsley's keynote presentation (which is a Powerpoint document), or rather, the linked data embedded in it. This graph allows you to explore the slides in the presentation, the concepts it discusses, resources and photos it contains, people related to it and the concepts it mentions, etc.

Linked Data Planet

2008-06-13T12:45:00.000-07:00

Next week is the first annual Linked Data Planet conference, which will be held in NY. I was really excited when I first hear about this, and excited about attending, because two of the keynotes are visionaries who I have been wanted to hear speak for such a long time but haven't yet had the opportunity: Kingsley Idehen and Tim Berners-Lee. I'm really excited about this particular event, because it puts a concentrated focus on the momentum building around Linked Data, which is one of the chief byproducts of the Semantic Web. I believe that this event will mark a critical turning point for the Semantic Web movement.

I will also be doing a talk on Dbpedia, Ontowiki, and Cypher, and a new service called Cynapse. In addition, I will have a demo of some of the latest Cypher features and improvements both in presentation and in the exhibition.

Poweset hype, and Norvig pooh pooh's on NL

2008-05-12T16:05:00.000-07:00

I saw a post this morning about Peter Norvig's remarks a few months ago about his perceptions of NL, and how it's all but useless in providing value to web search. The post resurfaced during this weekends' buzz over Powerset. Here's my reply.

I believe the discussion around Powerset and its potential suitors is on a misguided trajectory. A few months ago, Peter Norvig stated that NL provides only marginal advances over the state-of-the-art keyword search technologies, and that key word lookup is actually more natural for users than NL questions and phrases. As a NLP advocate in general, and a die-hard advocate of knowledge-driven NLP, I am amazed to find myself in perfect and absolute agreement with Norvig's assertions. A simple and concise list of keywords are the most suitable interface for search and retrieval of text documents from the WWW.

But the focus on document search as the future of information retrieval is itself a fallacy. Google's blindspot, and potential undoing, is the insurgent linked data web, or web of data, or semantic web, or web 3.0 (pick your favorite), which has been heralded in by Tim-Berners Lee. This vision will allow the web to consist primarily of structured databases comprised of graphs linked together by dereferencable, non-ambiguous URIs. For the data contained in any segment of this global graph, and the schema encapsulating the data, the convenience of having a consistent model for data exploration, and the notion of a fixed domain of discourse to guide UI designers will become a thing of the past. The user will no longer "search for a page using keywords", but will instead "lookup an entry by description". Any one "lookup" may span dozens of domains of knowledge/ontologies/schema, and will yield result sets of such breadth and heterogeneity as would defy any attempt at achieving the GUI consistency of Google's ranked list of links. People using this gloabal graph will search not for pages deemed relevant to a bag of words based on the consensus of the crowd. Instead, users will look up people, places and things, and links and relations of varying complexity between them, using unambiguous references to those entities. In order to perform these laserbeam-like lookups, users will demand to leverage the interface they have spent a lifetime mastering, a UI that is no less natural (in the task of expressing relationships between things) than the natural language user interface (NUI), where noun phrases and named entities will allow users to make reference to a set of URIs as expansive as the NL lexicon itself, while verbs, adjectives, relational nouns, prepositions and modifiers will offer users a broad and rich set of operators for describing the links between those URIs. There is a time and a place for every purpose under heaven, and I believe this is the proper place for NL technologies. NL and the SW shall evolve together, and each will symbiotically facilitate the critical mass adoption of the other.

I believe every contributor to NL should be involved in a project which seeks fuse the semantic web with NL.

Cypher 1.2 Release, and Cypher Web Service

2008-04-26T06:24:00.001-07:00

I'd like to announce the 1.2 release of Cypher and the availability of an updated user guide. This release is also accompanied by the release of Cypher Web Service, which allows Cypher to run as a RESTful web service. A public demo of the web service is available.

Below is the change log:

feature enhancements since 1.1.8
- added report.xml file which is reported when cypher.report.html is true, file contains an xml version of the report in report.csv
- added cypher.report.html.refresh to control refresh time in HTML interface
- added cypher.input.files to control whether cypher.input.dir directory will be crawled
- added cypher.output.commit to control if RDF output will be loaded into cypher.repository.output
- added cypher.output.format to specify serialization for RDF output
- added cypher.http.base used for namespace of minted URIs and also the URL of the Cypher web service

#1 in SPARQL Software, LOL!

2008-04-23T13:03:00.000-07:00

I was looking at the traffic logs for monrai.com, and saw quite a few vistors from the Google results for sparql software, so I took a look. Turns out, my site is the first result, as well as in the other first few results. My question is, how did that happen?? I also took a look at Google Trends for these search terms, but there isn't enough data on them to show up on its radar.

I don't really know a lot about SEO, but something I did must have worked, because there are so many other projects, software, etc that are far more popular than Cypher, Cyparkler, etc. I would tell you to just Google sparql software, but... :) Just goes to show, that Google page ranking isn't 100% accurate 100% of the time.

Here's a few of the (truly) most popular:
Openlink Virtuoso
Sesame
Redland
Jena
Longwell

Hello Again World!

2008-02-28T11:32:00.000-08:00

I'm pleased to announce that after a lot of hard work and input from users and developers, Monrai Cypher beta release is a few days from being released. It is based on Sesame 2, features an entirely new lexicon and framenet based on RDF. It will be accompanied by a hosted service which allows users to set up their own Cypher instances online to eliminate the need to stand up your own server. So stayed tuned for the upcoming announcement.

Many thanks to Openlink and their team for all the support they have given to the project.

Been Busy

2006-10-04T13:57:00.000-07:00

I've been a bit skimpy on the post lately as I'm currently kicking off a new semantic web venture. I'm very excited about this venture first because nothing is more thrilling to me than launching a new start up, and second because the service is something that I could find myself using today and quite frequently, as well a lot of other people I know, that coupled with the fact that it's all based on semantic web technologies and concepts, well now you understand why it's managed to command so much of my attention. Expect a beta release announcement in the next 30 days. Also, I've noticed that the interest in Cypher is still very steady, and I'm starting to see the first wave of developers who are learning the Cypher techniques and experimenting with this stuff in the lab. Please feel free to contact me with any technical questions and I can usually find a second to help you out :))

Stay tuned...

Centralized Approach Revisited

2006-09-02T20:22:00.000-07:00

Here is a comment I posted to Nova Spivack's blog concerning Radar Network's recent announcement concerning the scalibilty of their semantic database indexing technology:

For those of you who don't know, part of our system is a homegrown distributed grid server architecture for massive-scale semantic search. It's not the end-product, but it's something we need for our product. It's kind of our equivalent of Google's backend -- only semantically aware. Like Google, our distributed server architecture is designed to scale efficiently to large numbers of nodes and huge query loads. What's hard, and what's new about what we have done, is that we've accomplished this for much more complex data than the simple flat files that Google indexes.

I've reposted my comments here for archive purposes:

A few weeks ago, I blogged about how little confidence I had in centralized approaches to semantic web database building. Giovanni Tummerello (dbin.org) wrote a great paper on the subject, and let me tell you, it's one challenging undertaking. The main challenge facing any centralized approach is what's known as the computational burden problem:

"On the WWW, the interaction is based on HTTP requests/replies that in the great majority of the cases will be of limited impact on the server (e.g serving a file). This means that, disregarding anomalous cases, both the computational resources and network traffic required by a HTTP request are bounded. On the contrary, “requests” on the semantic web are naturally expressed in query languages and, given the graph nature of RDF structured information, the complexity of execution is not bounded a priori as it is a function of the query type as well as the quantity and the structure of the data. In other words, whoever would decide to offer the ability to answer “arbitrary questions” on a SW, would easily open himself to “denial of service” situations even in the ideal, good faith usage."

Creating a centralized database that solves the computational burden problem is one of the holy grails of the semantic web. My hat goes off to you and your team for tackling and solving this problem. I always predicted that P2P networks were the only feasible solution. Giovanni's approach is to periodically synchronize each peer's database, but only from within small peer groups, and once the data has been downloaded the query is sent to the local database, thus limiting the "damange" to the user's local resources. The obvious drawback is that no one peer has 100% visibility across the entire distributed database. So if the answer to a particular SPARQL query happens to exist in triples across seperate peers, and I haven't sych'd with each of those peers or I'm not in those peers' groups, then I'm just up the creek. The ideal repository would be centralized, and accept SPARQL with the speed and scaliblity of Google, which (correct me if I'm wrong) sounds to me you guys have achieved. Again, I'm jaw dropped. For example, this will have serious ramification for my work with Cypher, as my major Achilles Tendon is the lack of a centralized repository of shared lexical descriptions (in RDF) collected from across the semantic web. If your service/framework could crawl, collect and most importantly "cook" RDF lexical descriptions (as the last item is what's lacking in current services like Swoogle), and if it can serve Cypher results to arbitrary SPARQL which queries the metadata of lexical entries, then you've just sped up natural language processing for the Semantic Web by about 5 years!

Sunday Surfing: Java for .NET

2006-08-27T11:56:00.000-07:00

A friend sent me a link to the IKVM framework a few weeks ago, and as the week winded down, I was finally able to look more into it. For you Microsofties who build on .NET, and for the Java developers looking to interoperate with the Microsoft development world, IKVM looks to be a great solution. It provides a VM implementented in .NET, and Jave core class libraries implemented in .NET. The payoff is that .NET applications can leverage Java libraries, and visa-versa. There are of course other ways of interoperating, but this approach really allows for tight integration, which is sometimes nessassary in a integration project. There's no support for AWT/Swing, but I'm guessing 99.9% of the developers looking at this don't care. There is a potential project comming up in which I may get to use this stuff in at least a prototype environment, so I plan to post the results and experience.

Radar Network Shows Some Skin

2006-08-27T11:18:00.000-07:00

Nova Spivack's new venture, Radar Networks, is finally preparing to reveal the new and highly secretive (Web 2.0/Semantic Web/Meshup/PIM ???) project they've been working on for the last few years. I am really excited to hear they've gotten so far along in development, and am ancipating hearing just what this new technology platform their building is. More importantly, what will be its impact on the Semantic Web (and ergo Cypher):

...something happened that changed my mind about this recently. I had lunch with my friend Munjal Shah, the CEO of Riya, who has an investor, Peter Rip, in common with me. Listening to Munjal tell his stories about how he has blogged so openly about Riya's growth, even from way before their launch, and how that has provided him and his team with amazingly valuable community feedback, support, critiques, and new ideas, really got me thinking. Maybe it's time Radar Networks started telling a little more of its story? It seems like the team at Riya really benefitted from being so open. So although, we're still in stealth-mode and there are limits to what we can say at this point, I do think there are some aspects we can start to talk about, even before we've launched. And besides that our story itself is interesting -- it's the story of what it's like to build and work in a deep-technology play in today's venture economy.

Good to hear another Semantic Web company has found backing in the venture capital community. I'll be staying tuned.

RDF Radar and PingtheSemanticWeb

2006-08-22T07:22:00.000-07:00

The creator of PingtheSemanticWeb.com has a post about a new Firefox plugin for detecting RDF on the web:

One of the new comer is the Semantic Radar wrote by Uldis Bojars. This plug-in for FireFox will notify you if it finds a FOAF, SIOC or DOAP RDF document on the web pages your surf.

The characteristic of semantic web documents is that they are not intended for humans, but for software agents (like search engines crawlers, personal agent software like Web Feed Readers, etc). The consequence is that humans do not see these documents, so no body really knows that the Semantic Web is growing and growing on the current Web.

This is the purpose of this new Semantic Radar: unveiling the Semantic Web to humans.

The Semantic Radar: much more than that

This plug-in is much more than that. Effectively, each time it detects one of these semantic web documents, it will notify PingtheSemanticWeb.com web service.

This is where the interaction between semantic web services and applications are starting to emerge. Now Web browsers will detect semantic web documents and notify a web service acting as a central repository for semantic web documents

I had the thought to extend Cypher to query the PingtheSemanticWeb.com service to detect Cypher datasets, and to notify when it has loaded new datasets created by the user. My question is, is there a way for my software to detect only the RDF documents it is concerned with ( i.e. Cypher dataset documents)? If so, I think developing a simple ontology that can be used to wrap Cypher dataset documents into, basically to point to their location on the web and other metadata, then having Cypher to download the datasets would be an excellent project.

50K Euro Compression Prize

2006-08-15T09:40:00.000-07:00

Marcus Hutter has announced that a 50K purse will go to the developer of an algorithm which can compress the first 100MB of Wikipedia better than its predecessors:

Being able to compress well is closely related to intelligence as explained below. While intelligence is a slippery concept, file sizes are hard numbers. Wikipedia is an extensive snapshot of Human Knowledge. If you can compress the first 100MB of Wikipedia better than your predecessors, you(r compressor) likely has to be smart(er). The intention of this prize is to encourage development of intelligent compressors/programs.

If anyone wins the prize using Cypher's impeccable pattern-matching capabilities, we'll humbly accept your gratitude :)

More from Ebiquity Blog.

Release 0.7.2

2006-08-15T08:24:00.000-07:00

A new Cypher release is available. This is a bug fix release:

Version 0.7.2

Fixes: from 0.7.1

-- Hardcoded reference to smonroe login for Sesame server now removed

With previous versions, users had to create a Sesame account to match the account in the Cypher config file. This fix allows users to change the config file to match their own Sesame login info.

Update:

There was also a hardcoded reference to the two default Sesame repositories which was also found and fixed.

Centralized Approach?

2006-08-14T08:48:00.000-07:00

I ran across a centralized RDF search engine. Swoogle. From the site:

Swoogle has a collection of over 1M error-free RDF documents collected from the Web and an additional ~700K documents that have embedded RDF, are malformed but appear to be RDF, or are no longer accessible. We’ve intentionally limited the number of simple RSS and FOAF documents in the current collection.

A centralized database has obvious benefits, in an ideal world, a Google would crawl RDF documents and serve up queries through one central interface. But RDF isn't HTML, nor does SPARQL lend itself to any sort of straight-forward keyword mappings. Building a centralized database to process billions of open-ended queries per day is a mammoth undertaking. It appears that Google, who perhaps is the only company on the planet with enough imagination, incentive, and expertise to effectively build such a centralized database, is also the company who is most skeptical about the viability of the Semantic Web. The Semantic Web may also pose inherit threats to Google, who has built its empire on algorithms which attempt to address the deficiencies of the unstructured World Web Web.

I therefore believe that the path of least resistance for bootstrapping the Semantic Web will be a P2P network, or at the very least, a hybrid between the two. Swoogle seems like a great first attempt, and I'll be watching out for progress made by this and other centralized attempts, but I'd sooner bank on distributed P2P approaches.

Gartner's Hype Cycle

2006-08-14T08:34:00.000-07:00

The industry research firm Gartner has announced its Emerging Technologies Hype Cycle for 2006, which analyses the maturity, impact and adoption speed of 36 technologies and trends over the next ten years. Among this year’s themes of technologies eliciting significant momentum is the Semantic Web. The list includes new or heavily hyped technologies, where organisations may be uncertain as to which will have most impact on their business.

Release 0.7.0

2006-08-10T16:58:00.000-07:00

A new release of Cypher is available. This is a feature enhancement release. Now Cypher can generate the integer representation of any arbitrary natural language number:

Version 0.7.0

Enhancements: from 0.6.9

-- added new NumberTranscoder_LITERAL; allows natural language numbers to generate integer representation, the integer is wrapped in RDF literals of type xsd:nonNegativeInteger and xsd:NegativeInteger, making it consumable for semantic web applications.

-- added new number pattern grammar example to exploit number transcoder

There are also a couple of new grammar definition files which cover natural language numbers in English e.g. Five hundred twenty eight million five. But extending them to cover numbers in other languages shouldn't be a problem. The extended example dataset covers numbers up to tresrigintillion (10^102 I think, but correct me if I'm wrong). Sense so many people have been waiting for an online demo, I plan to set up the number transcoder as an intermediate online demo, especially since the input set in this case is finite.

I will post a more detailed explanation of the new dataset most likely in an article to be posted on the main Monrai website. In the meantime, try starting Cypher and entering: Your Name is some long number, for example Chris is twenty two thousand forty nine. Then look at the output file. There should be a owl:sameAs triple near the top, and one object should be the number you said. The BE verb is set to output an owl:sameAs triple, but you can easily change it to set the subject's age ( e.g. myonto:age). Also, conjunctions are not covered by the number patterns I wrote, so nine hundred and two won't match, but nine hundred two will match. I leave as an exercises for the user, the task of extending the example number pattern grammar to cover conjunctions.

Natural language numbers are normally spoken as opposed to written/typed, so speech recognition systems are probably a more appropriate usecase for this dataset.

Have fun!

ConceptNet

2006-08-10T12:54:00.000-07:00

Remember the Open Mind Project? Well, I recently heard about a group at MIT has taken that commonsense database and created a .NET explorer as well as a Natural Language Processing framework. Here's more from the site:

The ConceptNet knowledgebase is a semantic network presently available in two versions: concise (200,000 assertions) and full (1.6 million assertions). Commonsense knowledge in ConceptNet encompasses the spatial, physical, social, temporal, and psychological aspects of everyday life. Whereas similar large-scale semantic knowledgebases like Cyc and WordNet are carefully handcrafted, ConceptNet is generated automatically from the 700,000 sentences of the Open Mind Common Sense Project – a World Wide Web based collaboration with over 14,000 authors.

There's alot of talk in the docs about it using Microsoft IronPython, which I suppose is a derivation of Python. In my opinion, such common sense databases are akin to an RDF instance database. So while these types of databases don't explicitly offer the type of information Cypher needs to perform language processing, Cypher could be used to populate and query these databases using plain language. In addition, some data, such as type hierarchies, can be extracted from these sources to help in build lexicons. You can expect more Cypher support of such common sense resources as they continue to gain momentum.

OpenCyc 1.0

2006-08-10T10:20:00.000-07:00

CycCorp has released OpenCyc 1.0. The Cyc system is a database of common sense assertions (e.g. rain is wet, grass is found outdoors). A couple of years back, I wrote a Cyc microtheory transcoder as a sort of toy application for Cypher. The system translated natural language descriptions, phrases and questions into microtheories in CycL and queries. But I couldn't get enough people interested to justify the work. Looks like I might be blowing the dust off that old code.

Here's more on the announcement from the OpenCyc website:

Release 1.0 of OpenCyc includes:

The entire Cyc ontology containing hundreds of thousands of terms, along with millions of assertions relating the terms to each other, forming an upper ontology whose domain is all of human consensus reality.

English strings corresponding to all concept terms, to assist with search and display.

A compiled version of the Cyc Inference Engine and the Cyc Knowledge Base Browser.

Documentation and self-paced learning materials to help users achieve a basic- to intermediate-level understanding of the issues of knowledge representation and application development using Cyc.

A specification of CycL, the language in which Cyc (and hence OpenCyc) is written.

A specification of the Cyc API for application development.

I'm trying to find a RDF view of the Cyc database that actually exposes the knowledge using RDF semantics, if anyone knows of one please let me know.

Release 0.6.9

2006-08-08T23:09:00.000-07:00

There's a new Cypher release availible:

Fixes: from 0.6.8

-- dynamic addition of FOAF entry for proper nouns not already entered in the database

Slashfacet Semantic Interface

2006-08-07T11:44:00.000-07:00

Today, I ran across Slashfacet, a generic browser for heterogeneous semantic web repositories. The browser works on any RDFS dataset without any additional configuration. The interface controls change depending on the data being viewed. Reminds me alot of Dbin. Here's the paper.

Modified 'Star' Lexeme

2006-08-06T21:38:00.000-07:00

I was testing the lastest example dataset release, and discovered the following input didn't produce output:

Tom Hanks stars in The Terminal

After investigation, I noticed there was no word sense for 'star' which accounted for the in preposition-object construction. So I added it, and now the following works fine:

Tom Hanks stars in the Terminal --> RDF
the movies that star Tom Hanks --> SeRQL

As a side note, the datasets for the two 'movie' examples covered in the Cypher User Manual page are still on the way. I discovered a bug in how nominal clausal modifiers which are missing both the verb and subject are processed. This affects the pattern Actresses who played in movies with Tom Hanks. As a quick hack however, I just treated the noun phrase as having one clausal modifer with two prepositional phrases, and it parses fine. In actually though, the last prep-phrase is attached to the noun head movies: movies with Tom Hanks. And this is actually an abbreviation for: movies that are casted with Tom Hanks. The big difference is that the framework works best when frame slots are filled by clauses (i.e. verb lexemes), not nouns. By expanding the noun phrase prepositional phrase into a clause, we can now goveren the semantics of the noun phrase prepostional phrase by just referencing a verb. So, instead of adding a new feature the the movie lexeme to cover each possible prepositional phrase complement, we just find a verb which governs the semantics, in effect, reusing other lexemes. A better explination of this is on the way. Please be patient as I update the lexicon definition language to address this phenomenon.

AOL's Data Collection

2006-08-05T16:57:00.000-07:00

eBiquity has a post on AOL's release of a large-scale data collection of natural language questions and answers, and includes 20K manually annotated queries:

AOL Research has released some interesting data collections, including:

20K hand labeled, classified queries
3.5M web question answering queries (who, what, where, when …)
Query streams for 500K users over three months (20M queries)
Query arrival rates for queuing analysis
2M queries against US Government domains

Additional datasets are promised in the future.

Would be nice if someone took time to evaluate the use of this data for Cypher.

Google's N-gram Set

2006-08-05T16:50:00.000-07:00

eBiquity's semantic web blog has reported Google's announcement that it will share a gigantic n-gram dataset generated from a corpus of one trillion words from Web pages. Google found 1.1B five-word sequences that appear at least 40 times and 13.8M words that appear at least 200 times. The dataset will be distributed by the Linguistic Data Consortium.

Google’s describes its motivation as follows:

We believe that the entire research community can benefit from access to such massive amounts of data. It will advance the state of the art, it will focus research in the promising direction of large-scale, data-driven approaches, and it will allow all research groups, no matter how large or small their computing resources, to play together.

I wonder if the data set will be annotated with linguistic information such as part of speech and word frequency. Sounds promising for semantic web data mining and hopefully will provide another corpus of data for Cypher and NLP.

Link Grammar Parser

2006-08-05T15:13:00.000-07:00

I got contacted by one of the guys working on the Link Grammar Parser at Carnegie Mellon. The approach is one of the few (open systems) I've come across which attempts to derive a true semantic representation using a linguistic knowledge-based approach. I plan to download and compare with Cypher, and post the results.

Language of Thought

2006-08-05T15:10:00.000-07:00

I thought this was a pretty cool peice of software. I haven't had a chance to play with it much, but I thought it might interest you:

i just visited ur site and think that the idea of understanding natural language and then translating it into a format suitable for the semantic web is a cool idea.im working on a project called Nelements that displays knowledge in the language of thought. in the future i was also planning to work on a Nelements translator that can translate natural language text into the language of thought.

Release 0.6.8

2006-08-05T15:01:00.000-07:00

Cypher release 0.6.8 is a minor bug fix release. Among the issues fixed are:

-- fixed bug which prevented some Containers element constituients from matching
-- fixed bug which caused the console to stop listening for input
-- changed NounTranscoder's handling of possessive noun constructs, now possessive noun phrases are SeRQL queries which merge with the enclosing phrase, to accomidate such phrases as "my homes in Hoston"

The release also includes some minor changes to the Hello World example data set.

Wiki for Cypher

2006-08-03T15:35:00.000-07:00

This week, I've been testing a new service Monrai will be lauching in a few weeks: A database of lexicons, grammars, framenets, and RDF ontologies for Cypher. In the spirit of Wikipedia, the site will allow anyone to add and edit content to a shared dataset. The dataset will be availible as a large zip file.

We may ask some of you to serve as beta testers before making the official announcement. More to come...

Transcography - Part 1

2006-08-01T08:20:00.000-07:00

Cypher is based on a sub-discipline of natural language processing called Transcography, which was developed by Monrai with the goal of merging the field of natural language processing with the increasingly popular Semantic Web movement. Transcography is the process of parsing the phrase structure of a natural language construct, and translating the grammar output into a semantic representation. The output of each NL construct is three things: 1) a URI representation of the NL construct, 2) a set of one or more subject-object-value triples involving the URI, and 3) the set of all triples produced by sub-phrases. So, Cypher views any and all lingusitic input as a URI + related triples. Knowing this is key to understanding why the Cypher lexicon is such a powerful NL resource.

As an example of transcographic output, consider the phrase: John's coach. The transcographic process produces a URI representing the phrase, for example: http://john.mysite.com/MrDouglass, and a set of triples representing the statements involved in the phrase:

{http://john.mysite.com/me} jo:isCoachedBy {http://john.mysite.com/MrDouglass}

Cypher leverages these triples to create either an RDF model or an SeRQL query. The mode of output is based on whether the NL construct is a clause or description, or if it's a noun phrase or question. The triples of sub-phrases are recursively merged to produce a root graph represeting the root NL phrase or clause. For example, consider: John's coach knows Martin. The URI produced will represent this clause (e.g. the URI of a reified RDF triple, or the URI of a semantic frame), and a graph containing:

{qv:node1} foaf:knows {http://john.mysite.com/MartinCrump}

The URI qv:node1 represents a SeRQL query variable of a SeRQL query which was serialized in RDF. This is because the phrase John's coach is a relational noun phrase, and thus, is anaphora reference. By re-constructing the SeRQL query for the variable (by following the links from qv:node1), and then executing the query, a program can retreive the resouce that represents John's coach at the time of the query. This technique is used because John may have a new coach at the time of the query. Transcography stipulates that any anaphora reference be represented by a query variable (linked to the RDF representation of the SeRQL query) unless the program is ready to apply the variable value (e.g. to presenting it to a human user in an interface).

The word transcography is the combination of transcode, which means "to convert media from one format to another", and -graphy which is "writing or text representation produced in a specified manner or by a specified process". Thus the literal meaning is "text transcoding". Knowledge representation frameworks used in the process include RDF and Frame Semantics.

Cypher - Alpha Release

2006-07-31T16:19:00.000-07:00

After five years of R&D, we're proud to present Cypher, a natural language transcoder for the semantic web. Documentation is sparse at the moment, and bugs are plentiful, so we're asking everyone to please bare with us as we transistion the framework from the lab to deployment.

First, to address alot of people's concerns. Our website is undergoing a massive overhaul as we prepare to focus our company's resources on building out and supporting the Cypher community, so there may be a broken link here or there. As for our company's description and mission, we have taken on the task of making natural language processing a commodity, by providing a free (as in beer) and accessible framework for building applications that leverage RDF and high-quality natural language processing. This manifest as the Cypher engine, and "datasets" developed by Cypher users and developers. These contain descriptions of grammars and vocabularies of any natural language. The engine is designed to take as input a natural language construct ( e.g. phrase, clause), and generate the RDF or SeRQL representation of it. It does this by leveraging the linguistic information in the datasets.

We are the first week of our release (and already over 1000 downloads :), so please bear with the lack of resources availible at this point (white papers, documentation, forums etc.). In the meantime, please feel free to send any comments and/or feedback you may have about our Cypher technology.

Through this blog, I will be sharing release information (which will be frequent at the moment), pulling back the covers on some advanced Cypher programming techniques, and much more. Stay tuned...