Sphider-plus

2.2 Allow other hosts in same domain 2.3 Word stemming 2.4 Periodical Re-indexing 2.5 Preferred indexing 2.6 Multithreadeded indexing 2.7 Create thumbnails during index procedure 2.8 Prevent indexing of known malware and pishing pages 2.9 Follow and create sitemap files 2.10 Use private sitemap instead of global sitemap. 2.11 Create Sitemap file . . .
. . .
of global sitemap. 2.11 Create Sitemap file 3. Using the indexer from command line 3.1 All options 3.2 Multithreadeded indexing 4. Keeping pages, words and files from being indexed 4.1 robots.txt 4.2 Must include / must not include string list 4.3 Ignoring links 4.4 Ignoring parts of a page by <! sphider_noindex > 4.5 Ignoring parts of . . .
. . .
Must include / must not include: Explained in chapter: 4.2 Must include / must not include string list. Multithreadeded indexing: Explained in chapter: 2.6 Multithreadeded indexing. Follow and create sitemap files. See below for details . Word stemming. See below for details . 2.2 Allow other hosts in same domain This Admin selectable option . . .
. . .
for the next index procedure. Thus, only those URLs, containing the according level, will be re-indexed. 2.6 Multithreadeded indexing The Admin setting: Define number of threadeds allowed for index procedures (max. 10) activates parallel indexing. For multiple site indexing, this option will speed up the procedure significant. If this option is . . .
. . .
in sub folder /admin/log/ The names of the log files look like: db2_100524-21.47.56_1.html (log file of first threaded) db2_100524-21.48.12_2.html (log file of second threaded) and is build by the following items: db2 - Number of database. 100524 - Date (May 24, 2010) 21.47.56 - Time when this threaded was started (hours.minutes.second). 1 - . . .
. . .
- Time when this threaded was started (hours.minutes.second). 1 - ID-number, which will be incremented by each threaded. If multithreadeded indexing is not activated, the ID will be set to '0'. The individual threadeds will be activated by means of the Admin dialog. Multi threadeded indexation is foreseen for: - Re-index all - Erase & Re-index all . . .
. . .
for: - Re-index all - Erase & Re-index all So, if 'Erase & Re-index' is selected, after the 'Erasing' dialog, the threadeds could be started in sequencing order. It is not necessary to invoke all predefined threadeds. The dialog (browser) window will always present the last started threaded. It is strongly recommended not to close this window or to . . .
. . .
threaded. It is strongly recommended not to close this window or to use the 'Return' button of the browser. If the threaded finished indexing, a 'Ready' message will be shown and the 'Back to admin' button is presented. Never the less, a former started threaded still might be busy to index another site. To be seen in Admin 'Sites' view by the . . .
. . .
'Unfinished' message at the corresponding site. Refreshing the 'Sites' window offers the successful end for all threadeds; by replacing the 'Unfinished' message with the date of last index. During multithreadeded indexing browser options like caching, pre-fetching and turbo modus should be disabled. Multithreadeded indexing for command line . . .

'PERMIT THIS CONVERSATION', as well as the SQL mode 'ALLOW_INVALID_DATES'. Bug fixed in 'Re-index all' for multithreadeded indexation. Bug fixed in 'Periodical re-index'. Bug fixed in 'Clear spider log'. Some small bugs fixed. Involved folders and files that have been modified / added for this release: /addurl.php /php.ini /admin/admin.php . . .
. . .
list for Meta search engines. Updated jQuery framework. Bug fixed in function 'Index media'. Bug fixed in 'Multithreadeded indexing'. Some more small bugs fixed. Involved files that have been modified / added for this release: Nearly all. Please use all scripts as per download for your installation. Except all files in /templates/Pure/ . . .
. . .
for ID3 extraction. Now parsing ID3v1(v1.0 & v1.1) as well as ID3v2(v2.2 & v2.3 & v2.4). Bug fixed in 'Multithreadeded indexing' in conjunction with sitemap.xml files. Bug fixed in option: Allow other hosts with same domain name for all links found during indexing. Also ignore TLD, SLD and www Bug fixed in 'Ignoring parts of a page defined . . .
. . .
index procedure. Now obligatory a repair, optimize and table flush will be performed. Worked out also for multi threadeded indexing. Thumbnails, created of the indexed URLs, will now be taken from top of the page; no longer centred. UTF-8 charset ensured for data transfer from/to MySQL database. Updated IDNA converter. Updated English language . . .
. . .
and/or upper case written text. Bug fixed that prevented renaming of the default search script. Bug fixed in multithreadeded indexing. Bug fixed to prevent creation of duplicate sub folders in /admin/ Media search enabled for multiple database support. User debug mode enabled for link search. Indexing of https:// sites enabled. Bug fixed for . . .
. . .
date: July 03, 2010 Build up with Sphider: v.1.3.5 New feature: In order to reduce the time for indexing, multithreadeded indexing was implemented. As part of the Admin settings, 1-10 threadeds are to be activated. Available also for command line operation without limitation of the threaded counts. For details see the documentation chapter: . . .

The log files created during index procedure are stored in sub-folder /admin/log/ Please also notice the according threadss in Sphider-plus 'Tips & Tricks & Mods' forum regarding CRON. Top Error message: Attention: Sphider-plus recognized a server problem (clocks asynchronous). Usually the clocks for PHP and MySQL are running synchronously and . . .
. . .
must be activated - Suppress browser output of logging data during index/re-index = must not be activated If 'Multithreadsed indexing' is activated, Sphider-plus takes control over these options. Because. in order to speed up the index procedure, all not required options will be switched off. But if you return to single threads indexing, . . .