Dr. Website
Making Framed Sites User-Friendly for Search Engines
Dear Dr. Website: How do search engines index pages from Web sites that use frames? If a user is sent directly to one of the pages in my framed site, they won't see it in the context of the other frames that are supposed to be there. I want the documents in my Web site to be indexed, but I don't want the search engines to point directly to these pages.Most search engines have a lot of difficulty indexing framed sites. They don't navigate a framed Web site the way you intend your readers to--for example, by clicking links in a table of contents to update a prominent main frame. Search engines will ignore all information that's placed in <FRAME> or <FRAMESET> tags, so the best way to make information available to them is the same way you might show consideration for users with browsers that don't support frames. These users and the search engines will only be able to make use of information you've enclosed in <NOFRAMES> tags.
On a more positive note, some search engines rely on information in <META> tags placed in the header of your document to catalog information about your site, so supplying <META> information could help some search engines pick up your pages even if they can't parse the rest of your pages. Not all search engines make use of <META> information, though. A Webmaster's Guide to Search Engines and Directories offers detailed strategies for making your framed site more user-friendly to search engines, with a walkthrough of what a search engine might see if you don't provide non-framed alternatives.
If you can identify certain pages on your framed site that you don't want search engines to pick up because they might not make sense out of context, you can utilize the conventions that exist for blocking robots from accessing any given page or directory on your site. This requires you to create a file named robots.txt in the root directory of your Web site. In it, you would include the file name (with the associated path name) for each framed page that should be excluded. Webcrawler provides detailed information about Web robots and implementing a robots.txt file.
Site Content Ratings
Dear Dr. Website: I received a complaint from a customer who wasn't able to get into my site. He was using a content filtering program that blocked access to my site because it didn't have a rating for my site. Are there ratings, like for the movies, that I need for my site? As you've discovered firsthand, some users may have settings on their browsers or content filtering software that will block them from accessing your site just because your site doesn't have a rating from that service yet. It can be worthwhile to obtain ratings for one or more leading rating services even if you're sure no one would find your site's content objectionable. In addition, some emerging applications will make use of PICS for advanced searching or personalized information delivery as well as filtering purposes. SafeSurf and the Recreational Software Advisory Council are among the leading rating services; the W3C lists a number of such services.Many browsers and Web server software programs are beginning to incorporate support for PICS technology, a standard that can be used to create customized online content filters. There are a growing number of rating services that have their own proprietary ratings systems, mostly based on PICS, for labeling content with very specific criteria that are important to them.
You should visit the Web site for each rating service you want to sign up with. Each will have an online form you'll need to fill out in which you describe your site's content. After you complete the service's questionnaire, the form will generate the HTML code for your PICS rating. Check with your Web administrator or ISP administrator to see if their server software supports transmitting PICS labels in HTTP response headers; if it does, your visitors' clients will find them there. Otherwise, the browser will look for a <META> tag with a PICS label in the HTML it received. You would then need to include the <META> tag with your PICS rating in the header of each page on your Web site. search: CONTENT, SEARCH ENGINES
Reprinted from Web Week, Volume 4, Issue 5, March 17, 1997 © internet.com Corporation All rights reserved. Keywords: Date: 19970317
