CSCI

CSCI.6500 Distributed Computing over the Internet

Spring, 2007

Programming Assignment 2.

This assignment is to be done either individually or in pairs. Do not show your code to any other group and do not look at any other group's code. Do not put your code in a public directory or otherwise make it public. While you may get help directly from the instructor, you are encouraged to use the WebCT Discussions page to post problems so that other students can also answer and see the answers.

The goal of this assignment is to create an open, dynamically reconfigurable Web search engine using the SALSA programming language.

The search engine should be started with a UAN, an initial UAL, an initial URL, and a depth search level. Your engine should fetch the initial URL, indexing word occurrences in that document and producing a list of URLs to fetch in the next iteration. It must keep indexing documents until the depth search level is reached.

Your search engine can be queried at any point, even while building the index. A query contains a single word. The search engine should return the URLs for the documents that contain the word sorted by the number of occurrences of the word in each document. A search engine client may want to produce HTML to display those results.

To support fault-tolerance, your search engine can be requested to move to a different machine. This can happen at any point, for example, while building its index or responding to a query. Migration should be transparent to the clients, i.e., there should be no changes to the code of the search engine clients.

Possible Extensions:

To support load balancing, you need to create a meta-engine. The meta-engine is started with the names of two or more search engines (as defined above) and it is able to combine their query results into a single result. When you query the meta-engine, the meta-engine queries the search engines that it represents (independently of their location, since it is just a client from their perspective) and combines their results into a single result. A client of the meta-engine should not be able to tell the difference between a meta-engine and a normal search engine (the interface should be the same).
Make the search engine decentralized/peer-to-peer.

To learn more about SALSA, see OOPSLA 2001 paper. To download and install SALSA, see http://wcl.cs.rpi.edu/salsa/

Submission:

The due date for this project is April 3rd, 2007, 11:55pm EST. You should use the assignments drop-off box located at the course's WebCT page. Upload a zip/jar file containing all the relevant documented files, along with a README file containing instructions on running the program and explaining any design decisions you made.