Database Expert Needed To Build Custom In-House Search Engine

This project is pretty straightforward - help build a search algorithm that finds specific items on websites, logs the found page in a list, stores the home page via a compression engine for quick but compact retrieval, and controls on back end for how many threads and index / reindex controls. There will be a list of keywords to start and other search results will be used just to determine the first million or so pages to index, beyond that there will be logic used to determine what to index. Essentially this is a scraper dressed up as a search engine but the results will go into a big database that will utilize compression (something similar to Full Text Indexing but higher compression and speed) and search queues. We will need your opinions on the size and infrastructure needed to handle a small amount of search (the first 10 million indexed sites). Data stored in the database will be cross-referenced against several other search engines and category tables so that searches and type of site being searched are logged along with that sites links and homepage source code. Ability to quickly search through results a must. Search queries will vary and may include illegal characters so knowledge of proper indexing of such characters and searching algorithmically through such characters is a must - if you understand what I am talking about you are most likely qualified to avoid this problem. Experience coding DATABASES, SEARCH INDEXING and COMPRESSION ALGORITHMS a MUST for this project. Language and logistics will be whatever your strengths are - but DEEP DATABASE knowledge is a MUST.