- Index summary: page count, date of last update, and list of start URLs
- List of URLs for the documents in the index
- The meta data for each URL, including the date (as interpreted by the search engine), the page title, description, keywords, and key phrases.
- The link structure of the index. For each document in the index, it lists the documents it points to and the other documents pointing to it.
- The broken links encountered during spidering.
- The PDF files ignored. Sometimes PDF files are used as a container for scanned documents. The scanned documents are pictures to the search index and so are not scanned. To save download bandwidth, these image PDF files are not reread.
Showing posts with label webmaster tools. Show all posts
Showing posts with label webmaster tools. Show all posts
Tuesday, July 29, 2014
Reporting tools for viewing an index reorganized and enhanced
We have reorganized and expanded the tools available for viewing a search index. All of the tools are now accessible from the Retrieve details about the last update of an index link in the Actions section of the Account Settings page. These tools are available:
Labels:
broken links,
link structure,
meta data,
webmaster tools
Monday, March 25, 2013
Expanded tools to analyze search indexes
We have been expanding and improving the tools you can use to understand a search index. This note discusses three of these tools, accessible from the "Actions" section of your Search Configuration page by following the link Retrieve the list of URLs, meta data, and hyperlinks in an index.
You may notice more URLs in the "List of Hyperlinks" than in the "List of URLs". During indexing we carry out a more complete duplicate removal than that implemented during spidering. As a result, the spidered list may include pages that are removed during indexing.
You may also notice that the Starting URLs in the "List of Hyperlinks varies depending on whether the last spidering was complete or incremental. On a complete respidering, the starting URLs are just those specified in the "Include List" for the index. For an incremental spidering, the starting URLs are all the URLs in the current index.
- The list of URLs is extracted directly from a search index. It tells you the exact source documents that were used to generate the index.
- Meta data includes such items as the title for a page and the date the page was last modified. The meta data is shown for each of the URLs in the index; that is, it is the same URL list shown by the "List of URLs" tool.
- The list of hyperlinks from each page comes from the spidering log and it includes links both on and offsite. It tells you what links are on a page and what pages contain a particular link. It can answer the question "Why is this page in the index?".
You may notice more URLs in the "List of Hyperlinks" than in the "List of URLs". During indexing we carry out a more complete duplicate removal than that implemented during spidering. As a result, the spidered list may include pages that are removed during indexing.
You may also notice that the Starting URLs in the "List of Hyperlinks varies depending on whether the last spidering was complete or incremental. On a complete respidering, the starting URLs are just those specified in the "Include List" for the index. For an incremental spidering, the starting URLs are all the URLs in the current index.
Subscribe to:
Posts (Atom)