"Fossies" - the Fresh Open Source Software archive

Member "dpsearch-4.53/doc/dpsearch-clones.en.html" of archive dpsearch-4.53.tar.gz:

Caution: In this restricted "Fossies" environment the current HTML page may not be correctly presentated and may have some non-functional links. Alternatively you can here view or download the uninterpreted source code. That can be also achieved for any archive member file by clicking within an archive contents listing on the first character of the file(path) respectively on the according byte size field.

DataparkSearch Engine 4.53: Reference manual
PrevChapter 3. IndexingNext

3.5. Clones

Clones -- are documents having equal values of Hash32 on all document sections. Indentical copies of the same document always have equal values of Hash32. This allow to eliminate duplicate documents in a collection. However, if only title section is defined in sections.conf, all documents with different bodies but with identical titles will be considered as clones.

3.5.1. DetectClones command

DetectClones yes/no

Allow/disallow clone detection and eliminating. If allowed, indexer will detect the same documents under different location, such as mirrors, and will index only one document from the group of such equal documents. "DetectClones yes" also allows to reduce space usage. Default value is "yes". 

DetectClones no

PrevHomeNext
StopwordsUpSpecifying WEB space to be indexed