Sphider-plus

Log [ Documentation Summary ] Preamble: The info presented here is valid only for the latest release of Sphider-plus. At present version 4.2024a published February 12, 2024 is the actual release. 1. Settings, customizing and statistics 2. Indexing 2.1 Various options 2.2 Allow other hosts in same domain 2.3 Word stemming 2.4 Periodical . . .
. . .
attacks 19.3 Admin backend protection against remote access 19.4 Log file reporting attempts to abuse Sphider-plus 20. Bound database 21. Suggest framework 22. Integration of Sphider-plus into existing sites 22.1 Integration into existing site by use of Sphider-plus templates 22.2 Embed the search engine into existing HTML code 22.3 The . . .
. . .
] 1. Settings, customizing and statistics If you want to change settings, behavior and design of Sphider-plus, you can do so by means of the Admin interface. There is a wide range of settings foreseen for Sphider-plus. Separated into different submenus like: Sites: - Add Site - Index only the new - Re-index all - Re-index only preferred . . .
. . .
user' - Backup / Restore - Copy / Move - Optimize Templates: In order to enable customer's integration of Sphider-plus into existing sites, HTML templates are prepared for Search form Text result listing Media result listing Most popular queries etc. Three different designs are offered, which may be selected in submenu 'Settings'. If the layout . . .
. . .
this mod, indexing is forced even if the page already has been indexed. Additionally this mod will Clear Sphider-plus database before the re-index process. It will leave the following untouched: - Categories - Query log - Sites and all options: spider-depth, last indexed, can leave domain, title, description, - URL Must include, URL must Not . . .
. . .
hosts with the same domain name and it also ignores TLD, SLD and www. If e.g. calling from http://www.sphider-plus.eu links like: - http://sphider-plus.eu (without www.) - http://www.info/sphider-plus.eu (additional subdomain) - http://www.sphider-plus.com (different TDL) - http://www.sphider-plus.tec.eu (additional SLD) will be followed if . . .
. . .
one is only following the links to other hosts, if the found links are redirected. 2.3 Word stemming Sphider-plus is offering language specific stemming algorithms for 15 languages: Bulgarian, Chinese, Czech, Dutch, English, Finnish, French, German, Greek, Hungarian, Italian, Portuguese, Russian, Spanish and Swedish To be activated . . .
. . .
selectable option is supplied by a web service with an individual key, which needs to be signed up by the Sphider-plus admin. The web shots will be presented as thumbnails with the size of 130 X 174 pixels as part of the text result listing, individual for each result. It should be taken into account that this option will slow down the index . . .
. . .
this option is not available for Intranet and all 'localhost' applications. In 'Sites' view of the Sphider-plus admin backend you may check ability of this web service. Also required time (seconds) to create one thumbnail is presented. So you may decide, whether you like to use this web service for all pages to be indexed. More details . . .

2). How to integrate Sphider's search form into existing pages? How to transfer former settings into a new Sphider-plus installation? Error message: "Attention: The configuration file for database 1 and the table set xyz does not yet exist." Error message: "Warning: set_time_limit() . . . " Error message: "Warning: set_time_limit() . . . " Error . . .
. . .
set is set to: 'xxxx'. Must be set to: 'utf8mb4_unicode_ci' " Error message: " Unable to create folder. Sphider-plus will not be able to store any log file. . . " Error message: " Logging option is set, but cannot create folder for logging files. (option 1)" Error message: " Logging option is set, but cannot create folder for logging files. . . .
. . .
Logging option is set, but cannot create folder for logging files. (option 2)" Error message: " Attention: Sphider-plus recognized a server problem (clocks asynchronous)." Error message: " Access is not granted to this admin backend." Error message: " No results for you (1) or (2)." Error message: ". . . greetings from the admin of Sphider-plus: . . .
. . .
for you (1) or (2)." Error message: ". . . greetings from the admin of Sphider-plus: You tried to flood Sphider-plus with queries. Bye for now." Error message: ". . . You are known to be evil by trying to flood the search engine several times. You are blocked now." Error message: " Up to now, there is no valid database defined for user . . .
. . .
search results I'm seeing the full text information repeated. Why? Receiving 'server error 500' on fresh Sphider-plus installation (option 1). Receiving 'server error 500' on fresh Sphider-plus installation (option 2). Receiving 'server error 500' on fresh Sphider-plus installation (option 3). In the addurl form, is there a way to remove "none" . . .
. . .
is fully indexed, but nothing is found. Result listing does not offer media results. Why does v.3 of Sphider-plus use the MySQLi interface? Why PDO interface is not implemented for database access? Why to use multiple databases and table sets? Why to store all user IP GDPR conform? [ Answers ] Shouldn't the spider follow 301 HTTP redirects? . . .
. . .
Why to store all user IP GDPR conform? [ Answers ] Shouldn't the spider follow 301 HTTP redirects? Yes, Sphider-plus follows 301 and 302 redirects. But it might be necessary to enable 'Allow to index other hosts in same domain' Details about this option are explained in documentation chapter 2.2 Allow to index other hosts in same domain In . . .
. . .
will be asked for the currently valid values, which all were set to 'admin' as per default download of the Sphider-plus scripts. In case you don't remember your actual values, please follow the next FAQ. Afterwards you may enter your new values for your individual usage. Top Unable to log in as Admin. Always re-directed to the log in form. Why? . . .
. . .
on the server, attention, as it encrypts sessions differently. Adding the following to the php.ini in the Sphider-plus root directory, will let you get usual access to the Admin backend suhosin.session.encrypt = Off; Unable to log into the database 'Configure' menu. Always re-directed to the log in form. Why? Verify that you really use the . . .

/admin/GeoIp.dat /admin/geoip.php /admin/index_media-php /admin/install_all.php /admin/install_sphider-plusphp.php /admin/install_v.1.8.php /admin/messages.php /admin/php.ini /admin/spider.php /admin/spiderfuncs.php /admin/thumbs/ (new empty folder) /converter/rss2html.php /converter/rss.html /converter/rss_parser.php . . .
. . .
tables and additional table rows in already existing tables. If you update from a former version of Sphider-plusphp, please run the /admin/install_v.1.8.php script. If you upgrade from original Sphider or install from scratch, you don't need to run this script. Its features are also included in the other installation scripts. Top [ Outdated . . .
. . .
files. The common word files should not be used, if 'phrase search' is the standard type of search. Sphider-plusphp will become problems to find complete phrases. Therefore, in Admin settings the use of the common word files may be activated / deactivated by a checkbox. For more details, please notice chapter Ignored words New feature: Use a . . .
. . .
in conjunction with selecting the destination folder for the sitemap files. ../ is the root folder of the Sphider-plusphp installation. If the charset of a page to be indexed / re-indexed is not detectable, the home charset as defined in Admin settings is used. Improved search function for non-Latin symbols. Search function enabled for queries . . .
. . .
to select the chronological order of result listing: - 'Most Popular Links ' on top. Activating this item, Sphider-plusphp will present the result listing in order of before learned link attractivity. Defined as those links with the best user acceptance (clicks). Additional items in Statistics overview: - Queries total - Link clicks total Additional . . .
. . .
/admin/admin.php /admin/configset.php /admin/install_all.php /admin/install_bestclick.php /admin/install_sphider-plusphp.php /admin/spider.php /admin/spiderfuncs.php /include/click_counter.php /include/commonfuncs.php /include/searchfuncs.php /include/js_suggest/suggest.php /include/js_suggest/SuggestFramework.js /languages/ all files . . .
. . .
/languages/ all files ..,/settings/conf.php Attention: Starting with version 1.6, Sphider-plusphp supports logging of 'Most popular links'. This item requires additional rows in 'links' table of the database. If you update from a former version of Sphider-plusphp, please run the /admin/install_bestclick.php script. If you . . .
. . .
(1 - 10 seconds). In order to prevent performance problems and memory overflow for large amount of URLs, Sphider-plusphp may clean resources during index / re-index. Selectable in Admin settings, this item periodical will: - Free memory that is allocated to unused MySQL recourses. - Unset PHP variables, which are no longer required. Define max. . . .
. . .
/admin/db_backup.php /admin/db_main.php /admin/install_all.php /admin/install_reallog.php /admin/install_sphider-plusphp.php /admin/messages.php /admin/real_get.php /admin/real_log.php /admin/real_ping.js /admin/spider.php /admin/spiderfuncs.php /converter/pdftotext /converter/pdftotext.script /include/captcha.tff /include/commonfuncs.php . . .

at result pages. To be defined in Admin settings. Enable user suggestion for new Url to become part of Sphider-plus database (addurl by user). To be activated in Admin settings, the user is enabled to suggest sites. - If enabled, a link at the footer of the result page leads to the suggestion form. - The user will have to fulfil 'Url', . . .
. . .
input, DNS availability and MX-RR validation of dispatchers account. Suggested Url will be stored in the Sphider-plus database until Admin decision. - Suggested sites are presented in Admin submenu 'Approve sites' so that the admin may decide to - accept - reject - bann - Result of decision will be mailed to the dispatcher (if selected in Admin . . .
. . .
actual data set. For index/re-index follow sitemap.xml (to be activated in Admin settings). If available Sphider-plus will use the sitemap to follow all links of that domain. This increases significant the speed for index and re-index. The mod will also force Sphider-plus to re-index only links that are: - New and not jet known in Sphiders link . . .
. . .
Sphider's 'last indexed' date. Search for part of a word by means of * wildcards. This mod enhances the Sphider-plus capabilities to search also for parts of a word. Invoke this mod by entering a * as first character of your search query. You may use * wildcards like: *searchme *searchall* *search*more* Search !strictly for the search query. . . .
. . .
the search query. Invoke this variant by entering a ! as first character of your search query. If you search for '!plus' only results for the word 'plus' will be presented in the result pages. No results for words that contain 'spider-plus' or 'spiderplustec' will be shown. This is the reverse function of 'Search for part of a word by means of * . . .
. . .
link-table but not in the stored keywords. The search output has the same look and feel as usual in Sphider-plus search results. Enabled search for dates like 2008-11-03 , 03/11/2008 or 03.11.2008 Enabled suggestion also for search queries that containing upper case characters. Automatically adapt Sphider's dialog to user language. This . . .
. . .
/admin/urls/ Import url.txt file from folder /admin/urls/ The content of file 'url.txt will be copied into Sphider-plus database. Existing Url's will be lost and overwritten. Following rules are valid for the url.txt file: - Url's must be in format: url,spider-depth,category like: http://www.abc.de http://www.abc.de,2 http://www.abc.de,-1,Info . . .
. . .
now are stored in individual files for each table. - Backup utility selectable for: 'Structure only' or 'structure plus data' - Unlimited file size for restore function is ensured. - Backup files compatible to phpMyAdmin. - Optimize Database. Bug fix by re-writing the complete Database Management. Links that do not contain page name are now . . .
. . .
does not except links like <a href= ?id=3 >link text</a> Thanks to the bug fix of BenRosey Sphider-plus follows correctly. Links that do not contain slash at the end of the Url are now correctly followed (Bug fix). Original Sphider does not except links like: http://www.abc.de Sphider-plus adds the required slash automatically . . .

/admin/configset.php /admin/spider.php /admin/spiderfuncs.php /admin/url_backup.php /admin/settings/backup/Sphider-plusdefault-configurationphp_default-configuration.php /admin/thumbs/.htaccess /include/categoryfuncs.php /include/commonfuncs.php /include/search_10.php /include/search_40.php /include/searchfuncs.php /include/search_links.php /include/search_media.php . . .
. . .
all files Version: 2.6 Release date: March 08, 2011 Build up with Sphider: v.1.3.5 In front of Sphider-plusdefault-configurationphp version 2.5 the following items have been added / modified: New feature: Result output is available now also as an XML file. If requested in search.php script, the results will be presented as XML file in /xml/ For details see . . .
. . .
the advanced search settings to OR. Complete redesign of all search files for easier integration of Sphider-plusdefault-configurationphp scripts into an existing HTML site. With special thanks for the suggestions, ideas and the participation of Carl Erling http://www.tba-berlin.de New Admin setting: Define whether the 'Search form' and the 'Result listing' of . . .
. . .
http://www.tba-berlin.de New Admin setting: Define whether the 'Search form' and the 'Result listing' of Sphider-plusdefault-configurationphp is embedded into an existing page HTML layout and design, or whether it is used as an independent page. For details see the documentation chapter: Integration of Sphider-plusdefault-configurationphp into existing sites New Admin setting: Define the . . .
. . .
Sphider-plusdefault-configurationphp into existing sites New Admin setting: Define the name of the search script in root folder of Sphider-plusdefault-configurationphp (default: search.php). Separated style sheet files are now included for Admin backend and for the User interface. This enables to individualize the User style sheet without destroying the Admin design. For details see the . . .
. . .
style sheet without destroying the Admin design. For details see the documentation chapter: Integration of Sphider-plusdefault-configurationphp into existing sites Improved 'Did you mean' algorithm. Now searching for a wider range of potential keywords. Break character inside of words will now be ignored, so that the complete word becomes indexed and searchable. Output . . .
. . .
/templates/Pure/userstyle.css /templates/Slade/adminstyle.css /templates/Slade/userstyle.css /templates/Sphider-plusdefault-configurationphp/adminstyle.css /templates/Sphider-plusdefault-configurationphp/userstyle.css Attention: This version requires an updated set of database tables. In order to create the new set of tables, run the 'Install all tables' for all databases in 'Database . . .

Sphider-plus is a search engine, based on the scripts of original Sphider. [ About Sphider-plus] More than 400 new features (additional mods, functions, template designs and debugging) have been added to the original Sphider. For details about all the improvements and changes, please read the Documentation section. [ Main features ] Item . . .
. . .
media links, User IP, Country code, Host name, Last queried, Top keywords, etc. GDPR support By default, Sphider-plus collects and processes user data in the G eneral D ata P rotection R egulation (GDPR) compliant manner. Segmentation of Chinese and Korean words Will divide phrases like 帽子和服装 into the base words 帽子 and 和 and 服装 , so that all . . .
. . .
🥵 👀 🥂 ⛔️ 🆗 ⏏️ 🕷 🕸 🧑 ‍🎄 and UNICODE symbols ‍ 🎄 👆🏼 ★ ⦿ ♜ ♞ ➨ ➤ ➽ 🂡 ☀︎ ✈︎ 🞧 ❗️ (Foreseen for next release of Sphider-plus) User may suggest URLs User may suggest new sites to become part of database. With Admin approval, reject and banned domains manager. Optional: suggested site needs meta tag with authentication code. Intrusion Detection System . . .
. . .
version 8.0.32 - MariaDB version 10.4.28 sp_executesql Ready to run in PHP 8 environment Latest version of Sphider-plus version 4.2024a is proven up to PHP version 8.3.2 [ Proven ] Successfully implemented as search engine on a customer site with a database capacity such as: 25.206 sites 324.595 page links 1.260.698 keywords 169.251 media links. . . .

Log [ Change Log Summary ] [ Outdated version ] Version v.2.1 Release date: September 03, 2009 In front of Sphider-plus version 2.0 the following items have been added / modified: New item in Admin settings: Perform a segmentation of Chinese and Korean text during index / re-index procedure. Will divide phrases like 帽子和服装 into the base words 帽子 . . .
. . .
charset: EUC-KR and ISO10646-1933 New item in Admin setting: Index password protected sites. If enabled, Sphider-plus will index also .htacces protected sites (basic authorization). Up to 3 different zones could be registered in Admin settings and will be indexed. New options in Admin settings: - Index framesets - Index iframes If enabled, both . . .
. . .
. . >; is detected in a sitemap.xml file, and if multiple Sitemap files are available, Sphider-plus will process the secondary Sitemaps and extract all links for index / re-index. Also gzip-compressed files (Index Sitemap files as well as the Sitemap files) will be processed. Improved index / re-index procedure: If charset of . . .
. . .
as defined in Admin settings will be used for the involved link. Improved index / re-index procedure: If Sphider-plus is relocated by http 301 or 302, links found at the relocated site will also be followed. For new sites, as per default the spider-depth is now set to 'full'. Improved UTF-8 support: Conversion into UTF-8 charset now is . . .

(1 - 10 seconds). In order to prevent performance problems and memory overflow for large amount of URLs, Sphider-plus may clean resources during index / re-index. Selectable in Admin settings, this item periodical will: - Free memory that is allocated to unused MySQL recourses. - Unset PHP variables, which are no longer required. Define max. . . .
. . .
/admin/db_backup.php /admin/db_main.php /admin/install_all.php /admin/install_reallog.php /admin/install_sphider-plus.php /admin/messages.php /admin/real_get.php /admin/real_log.php /admin/real_ping.js /admin/spider.php /admin/spiderfuncs.php /converter/pdftotext /converter/pdftotext.script /include/captcha.tff /include/commonfuncs.php . . .
. . .
/templates/all_folders/thisstyle.css Attention: Starting with version 1.5, Sphider-plus supports real-time output of logging info during index / re-index procedure. This item requires an additional table for the database. If you update from a former version of Sphider-plus, please run the /admin/install_reallog.php . . .

Release date: April 23, 2010 Build up with Sphider: v.1.3.5 In order to ease customer's integration of Sphider-plus into existing sites, HTML templates are prepared for - Search form - Text results - Media results - Most popular queries - etc. New feature: Allow indexing of other hosts with same domain name for links found during indexing. . . .
. . .
Split words into their basic parts, separated at each hyphen, dot or comma inside the words. For example 'sphider-plus.eu' will be divided into the 3 keywords: sphider plus eu As also the original word is stored as keyword, all 4 words become searchable. Alternatively the separation only at hyphens is selectable in Admin settings. New feature: . . .
. . .
the corresponding prefix. New Admin setting: Define directory for templates (relative to root directory of Sphider-plus) Search and open media files enabled now for media links with up to 1024 characters. Input settings for database configuration menus are now enabled for values up to 255 characters. 'Clean resources' improved for index . . .

information. Introduction Release and Legal Info Installation Documentation Change Log [ Release ] Name: Sphider-plus Version: 4.2024a Released: February 12, 2024 Based on original Sphider version 1.3.5, released 2009-12-13 [ Legal Info ] This program is licensed under the GNU GPL v.3 by Rolf Kellner [Tec], tec(a t)sphider-plus.eu Original . . .
. . .
that proprietary software companies offer: none. Imprint Privacy Policy [ Donation ] If you want to use Sphider-plus and also want to promote further development, your remuneration is most welcome. . . .