Searx-Qt

Desktop client application for Searx/SearXNG/searx-space

This is documentation for Searx-Qt version 0.5.0-alpha1

About

Summary

Search with Searx/SearXNG instances without the need for a complex web-browser.

Searx-Qt is a desktop application that lets you search on public SearXNG instances listed on https://searx.space (or you can add Searx or SearXNG instance urls manually).

The aim of the project is to preform search operations on Searx/ SearXNG instances with the use of their API and without the need for a complex web-browser. Searx-Qt will do the bare minimum of what is required to search, no cookies and no javascript. Just render a Searx/SearXNG API request to something usefull.

Unfortunately most public SearXNG instances actively block requests to their JSON-API. But don't worry! Thats why there is a option in the SearXNG settings (which is on by default) to parse HTML instead, and still no javascript and no cookies.

Since version 0.2 it is also possible to manage your own (private?) instances with the use of a 'user' profile.

Searx

The Searx project is no longer maintained but for now Searx-Qt will still support it.

API Docs: https://searx.github.io/searx/dev/search_api.html
Source: https://github.com/searx/searx
License: AGPL3
Status: Inactive

SearXNG

API Docs: https://docs.searxng.org/dev/search_api.html
Source: https://github.com/searxng/searxng
License: AGPL3
Status: Active

searx-space

The searx-space project lists public SearXNG instances with statistics. The official instance is running at https://searx.space/. This is where Searx-Qt will request a list with instances when the update button is pressed by default (the searx-space url can be changed in the settings).

Source: https://github.com/searxng/searx-space
License: AGPL3

Source

https://notabug.org/CYBERDEViL/searx-qt

License

GPL3 https://www.gnu.org/licenses/gpl-3.0.en.html

Dependencies

name	version	license	URL
python	3	PSFL	https://docs.python.org/3/license.html
requests		Apache 2	http://docs.python-requests.org/en/master/
jsonschema	>= 4.10	MIT	https://github.com/python-jsonschema/jsonschema
PyQt5		GPL3	https://www.riverbankcomputing.com/software/pyqt/intro
urllib3		MIT	https://urllib3.readthedocs.io/
beautifulsoup4		MIT	https://www.crummy.com/software/BeautifulSoup/

Optional for socks proxy support:

name version license URL

pysocks

BSD https://github.com/Anorov/PySocks

name	version	license	URL
pysocks		BSD	https://github.com/Anorov/PySocks

Building:

name version license URL

gettext

GPL https://www.gnu.org/software/gettext/

PyQt5-dev Qt5 GPL3 https://www.riverbankcomputing.com/software/pyqt/intro

name	version	license	URL
gettext		GPL	https://www.gnu.org/software/gettext/
PyQt5-dev	Qt5	GPL3	https://www.riverbankcomputing.com/software/pyqt/intro

Translations

The default language is English.

Since version 0.2 Searx-Qt is able (application-wise, not search results) to be fully translated with the use of gettext and .po .pot files. However the only translation available currently is for the Dutch language. If you like to translate Searx-Qt in your language then you can find a example of how to do that in the development section. Please consider opening a PR on https://notabug.org/CYBERDEViL/searx-qt after your translation has finished.

Contact

Please open an issue on https://notabug.org/cyberdevil/searx-qt.

You may also want to see if anyone is online on IRC at #searx-qt at irc.libera.chat:6697.

Getting started

Install dependencies

Note: python-requests is also dependent on python-urllib3 ; so python-urllib3 will be installed with python-requests (No need to do a explicit install).

Debian / Ubuntu based

Install required dependencies:

# apt update
# apt upgrade
# apt install python3 python3-requests python3-pyqt5 gettext \
              pyqt5-dev-tools python3-jsonschema python3-bs4

Optional for socks proxy support:

# apt install python3-socks

Arch based

Install required dependencies:

# pacman -Syu python python-requests python-jsonschema python-pyqt5 \
              python-beautifulsoup4 gettext qt5-tools

Optional for socks proxy support:

# pacman -S python-pysocks

Installation

It is always recommended to let the package-manager of your system do the installing of software, so your package-manager will keep track of files installed. Only use setup.py directly if you know what you are doing.

Since Searx-Qt isn't available in any GNU/Linux distribution (yet?); the best option is to create a package for your distribution yourself from the latest release. This will also mean that you have to manually update Searx-Qt if there is a new version available.

Note: https://notabug.org/CYBERDEViL/searx-qt/releases

Note: noticed the # or $ before every command? When there is a $ before the command, it should be run as a regular user. # as root.

Debian based

The steps below describes how to get a specific version of Searx-Qt; then package and install it. This method is available from version 0.1-beta2 and up.

Make sure you have python3-stdeb and git installed:
```
# apt install python3-stdeb git
```
Creating a working directory and cd in to it, you may change this to your own preference:
```
$ mkdir ~/git
$ cd ~/git
```

Cloning the repository and cd in to it:

$ git clone "https://notabug.org/CYBERDEViL/searx-qt.git" "searx-qt"
$ cd searx-qt

Checkout a specific version:

Note: get a list with available tags (versions) with the git tag command.

Below is a example to checkout version 0.5-alpha1:
$ git checkout 0.5-alpha1

Create .deb:
```
$ ./utils/gen_deb.sh
```

Install the created package:

# dpkg -i ./deb_dist/python3-searx-qt_0.5-alpha1-1_all.deb

Arch based

For Arch based distributions there is a package available in the AUR; https://aur.archlinux.org/packages/searx-qt/

Make sure you have git installed:
```
# pacman -S git
```
Creating a working directory and cd in to it, you may change this to your own preference:
```
$ mkdir ~/pkg
$ cd ~/pkg
```

Getting the PKGBUILD from Arch AUR:

$ git clone https://aur.archlinux.org/searx-qt.git
$ cd searx-qt

Build and install Searx-Qt package:
```
$ makepkg -si
```

Usage

Profiles

Profiles are useful when you want to have different settings and/or data without to having to set it manually every-time. For example you can create a profile named Tor which has different proxy and stats2 settings then you normal profile.

There are two types of profiles:

Stats2 profile
User profile

The profile type names maybe changed to something better, suggestions are welcome.

Create a Stats2 profile if you wish to get/update a list of Searx-instances from a Searx-Stats2 instance. For example the default https://searx.space.

Create a User profile if you wish to add/remove/update your own list with Searx-instances.

NOTE: Profile types cannot be changed after the creation of the profile,

but you can add multiple profiles of both types.

Creating new profile

On first usage of Searx-Qt you will need to create a new profile. The Add button (of the "Profile select" window) will open a dialog to do so.

There are profile settings presets (Web, Tor, i2p) which you can choose from. The Tor preset sets the proxy to 127.0.0.1:9050 and changes the Searx-Stats2 instance url to the onion address. The i2p preset sets the proxy to 127.0.0.1:4444, it also adds some known i2p instances of Searx.

Deleting a profile

I hope that it is self explanatory that the Delete button of the "Profile select" window deletes the currently selected profile, it will ask for confirmation before doing so.

It is not possible to delete a active profile (at-least it shouldn't ;-)).

Settings

General

Theme

A Theme is a Searx-Qt specific stylesheet and the Base style is a Qt theme/style.

The Searx-Qt specific Theme does override the Base style.

CLI output level

The amount of CLI spam can be set here.

Info does print to stdout.

Warning does print to stderr.

Debug does print to stderr.

Errror does print to stderr.

Connection

Verify (SSL)

Request will fail on a invalid SSL/TLS certificate.

Leave checked if unsure.

See https://requests.readthedocs.io/en/master/user/advanced/#ssl-cert-verification for a more technical description.

Timeout

Timeout in seconds for a single request.

Leave it at the default value of 10 seconds if unsure.

See https://requests.readthedocs.io/en/master/user/advanced/#timeouts for a more technical description.

Proxy

Here you can set a proxy that will be used for every connection Searx-Qt makes.

The set proxy will apply to both HTTP and HTTPS requests.

If you use a socks4 or socks5 proxy you probably want to make sure the 'Proxy DNS' checkbox is checked so DNS requests will also go through the proxy. DNS proxy is not available for a http proxy type.

User-agents

What user-agent string should Searx-Qt send?

After pressing the Edit button it will change to a Save button, you will be able to edit the user-agent ?string(s) Searx-Qt will send. Some notes:

One user-agent string per line.
Set total blank to not send any user-agent string.

When the Random checkbox is checked and there are multiple user-agent strings set then Searx-Qt will pick a random user-agent string from the list for every request.

Extra headers

Excluding the User-Agent string, these are the headers Searx-Qt will send. As well the header key as the value should between double quotes ("), the key and the value should be separated by at least one space. Edit these only when you know what you are doing!

SearXNG

Parse HTML

When this is enabled Searx-Qt will request HTML on search request and not make use of the SearXNG JSON-API. The HTML will be parsed and turned into a JSON object, what will be processed as usual.

This is enabled by default.

Safe Search

When this option is enabled it will simply send safesearch: "1" with a search requests. It is whatever SearXNG and the search engines it relies upon will do with it.

It is assumed that NSFW results are left out (filtered-out). So when left unchecked NSFW content may be returned, and when checked there should be none (BUT NO GUARANTEE).

This is disabled by default.

Searx-Space

Here you can change the URL of the Searx-Stats2 instance you like to use for fetching the instances data.

NOTE: This is only available for a Stats2 profile type.

Guard

Guard can put instances on a timeout or the blacklist when they are failing and is enabled by default.

When the Guard rules are set properly, searches will be quicker over time since failing instances are not used anymore. This reduces the chance of requesting a search query to a instance that probably will fail again. Also some instances block our request for whatever reason they have, and there is no standard response so they all may (and many will) respond differently (Thus we can not properly detect when we may use the API and when not).

It should be obvious that when Enable guard is checked that Guard is enabled, when not checked Guard is disabled.

When Guard is enabled it is advised to also enable the Store log option so that old failures can be evaluated against new failures after Searx-Qt has been restarted.

Below the Store log option is a spinbox that defines for how long log entries will be stored (in days).

A rule defines what a fail is and what should happen with the failing instance.

Rules

A rule has the following variables:

Error Type, the type of error.

Amount of fails.

Timeframe in which the Amount of fails have to occur.

Status, the HTTP response code. (Only used for the WrongStatus Error type).

Destination, what should happen to the instance? Should Guard put it on the blacklist or on a timeout?

Duration in minutes of the timeout. (Only used when Destionation is set to Timeout). When Timeout is used as Destination and this is set to 0 minutes the instance will be on timeout until Searx-Qt has been restarted.

When Error type, Amount, Timeframe and Status are met the rule will be triggered and the instance will be put on the Destination for Duration amount of time in minutes.

Log

Here you can see failed search requests. Failed search requests will only be logged when Guard is enabled.

It logs as little as possible. The following is logged:

Date and time.

Instance url.

Error Type.

HTTP status code.

HTTP content/Error message (for debugging, may contain error message which is handy for debugging but also may contain SENSITIVE stuff!)

Instances

A Searx instance is a server running the Searx project. Since we want to preform searches to Searx instance(s) we need addresses of those instance(s).

The interface to manage instances is on the right.

With Stats2 profile type

When your profile is a Stats2 type, the Searx-instances will be fetched from https://searx.space/data/instances.json (or any other set in the settings by your preference). The instances.json from search.space also contains a lot of other data about the instances it lists (which we can use to filter instances based on our preferences).

When Searx-Qt is used for the first time you will need to update the instances table. There is a 'Update' button between the Filter and the Table that can be used for this. Searx-Qt will not update this automatically!

It maybe useful to update the instances data so now and then since public instances appear, disappear and their stats change over time.

With User profile type

If your profile is a User type you will have to add addresses of instances manually.

This can be done by pressing the Add instance button right above the instances table, a dialog will pop-up asking for the address to add (without scheme).

The scheme (http:// or https://) can be selected from the combobox.

There is also a "Update data on add" checkbox, when this is checked and Add is pressed it will automatically download data from http(s)://your-address/config. Downloading/updating this data may also be done later by right clicking on a (or multiple) Searx-instances in the table and pressing Update selected from the context-menu that has popped-up.

Instances table

The instances table can be used to browse instances with their data that remain after all filters. The table is also used to set the current instance by left-clicking on one.

The currently used instance should also be visible bottom right in the application it's status-bar.

Right-clicking in the table opens a context-menu from where you can do the following:

Whitelist/blacklist selected instance(s).
Temporary blacklist.
Copy any column(s) of the selected instance(s) to the clipboard.
Copy JSON data of the selected instance(s) to the clipboard.
Select All instances (CTRL+A should do the same).
Hide or show columns.

If your profile is a User profile the context-menu will have the following extra actions:

Remove selected instance(s).
Update selected instance(s).

Filter instances

When a filter is enabled and the instance it's value that is being matched is unknown then it is excluded by default!

Network

Filter instances on network type. Only instances that match one of the checked network types remain.

Require ASN privacy

Excludes instances that run their server at a known malicious network. Like for example CloudFlare, Google, Akamai etc..

This does not give any guarantee, it only filters known privacy violators!

For a full list of known malicious networks (technical): https://github.com/dalf/searx-stats2/blob/master/searxstats/data/asn.py

Require IPv6

Exclude instances that don't have at least one IPv6 address.

Version

This has multiple options to filter on Searx/SearXNG versions strings. Both semantic versions and date versions are supported.

[Minimum] Include only instances with versions of the set minimum version or higher.

[Invalid] Include instances with a invalid version string.

[Development] Include development versions (git versions).

[Dirty] Include SearXNG development versions with uncommited changes.

[Extra] Include versions with the 'extra' addition, I don't know what this is; so TODO.

[Unknown] Include Searx development versions with unknown changes.

Blacklist

Here are the URLs of the instances that have been blacklisted, either manually or automatically by Guard (when enabled).

There is a button right to each blacklist item to remove it from the blacklist.

Hovering the remove button or the url of a blacklist item will show a tooltip with some more info.

You can manually blacklist a instance by right clicking on a instance in the instances table and click 'Add to blacklist'; multiple instances can be blacklisted at once.

Blacklisted instances will be excluded from the table by default.

Whitelist

Here are the URLs of the instances that have been manually whitelisted. There is a button right to each whitelist item to remove it from the whitelist.

You can manually whitelist a instance by right clicking on a instance in the instances table and click 'Add to whitelist'; multiple instances can be whitelisted at once.

Whitelisted instances will be in the table by default except when they are on the timeout list.

Timeout

This is a temporary blacklist. Instances manually put on a timeout will stay here until Searx-Qt is restarted. When Guard is enabled it also may put instances here depending on the set rules, those may persist after Searx-Qt is restarted depending on the rule(s).

Hovering the remove button or the url of a timeout item will show a tooltip with some more info.

Search

Search bar

Fallback

When checked it will pick a random instance from the instances table if a search request fails one way or another and re-try the same request with the freshly picked instance. There is a maximum amount of 10 tries (10 different instances to try the same request on).

What is fail?

Connection errors including timeout.
Wrong status code (not 200).
No (usefull) or malformed results returned.

Random every

When checked it will automatically pick a random instance on a search request, it will also hide the 'Random search button' because it makes it obsolete.

When not checked it will do search requests on the same instance unless the request fails somehow and 'Fallback' is checked. Exception is when the 'Random search button' is used for the search request.

Random search button

When pressed it will pick a random instance from the list and preform the search request.

Reload button

When pressed it basically preforms a search request without 'Fallback' whenever it is enabled or not, it also doesn't reset the page number. So it can act as a reload button thus it's name, but it does more.

Note: When a search argument like the search query, instance URL, categories/engines etc. has changed by user interaction it will do the request with those changes, that isn't a real reload of the previous request.

Dev-note: Probably this behavior should change or the name/icon should change to something more fitting.

Search button

Preform a search request on the currently selected instance.

Page number is reset, 'Fallback' and 'Random Every' options are honored.

Search query input

The query you like to search for.

See https://searx.github.io/searx/user/search_syntax.html for what is possible.

It will do a search request on enter key pressed, same behavior as when the 'Search button' has been pressed.

Search options

NOTE: Right clicking in (on the picture above) the dark area opens a context-menu where you can manage what options you want to be visible or not as shown in the image below.

NOTE: Left-click (mouse) on the Categories or Engines label will toggle the label collapsed/expanded state, to be able to reduce the height when multiple options are selected.

NOTE: Right-click (mouse) on Categories or Engines label will open a context menu with a option to uncheck all for convenience.

Engines

Here you can toggle what search engines should be enabled. It will automatically filter out all instances from the instances table that doesn't have at least one of the checked engines enabled. The checked engines will be send with a search request to a Searx instance with the enabled_engines param. You should only get results from engines that are checked.

If no engine is checked it means that it may return results of any engine in the list.

The list with engines is created with data from the instances table, so only engines are listed that are available from the instances table.

Period

Search period you like results from. Options are Last day, Last week, Last month or Last year.

Language

If you want results in a specific language than you can select one here. The set language will persist on restart.

Since Searx-Qt 0.3 there is a option to mark languages a favorites. Favorites will appear on the top of the combobox list so you won't have to scroll.

Adding a language to favorite can be done by hovering the language and pressing the spacebar on your keyboard. Removing a favorite works the same, hover the favorite language and press the spacebar on your keyboard.

NOTE: Not all engines have language support and not all engines

honor the requested language. Searx-qt does not (yet?) act on

this.

NOTE: When Default language is set that means the default language

of the instance.

Search results

Find text in results

The find widget to search text inside the results can be opened/focused by the keyboard shortcut Ctrl + F.

Shortcuts that may be used while the find text input is activated:

Return to find the next match.

Shift + Return to find the previous match.

Escape to close.

Troubleshooting

Searx-Qt won't start

When Searx-Qt fails to start the first thing you should try is to run it on the CLI and see if there are any error messages.

When you just updated to a newer version of Searx-Qt it can happen that the stored profile data is not compatible. Searx-Qt should be aware of this and notify you when this happend but this is NOT the case yet. So one thing you can try is to delete the local Searx-Qt config (usually in ~/.config/CYBERDEViL/searx-qt/). Sorry for losing your previous config, this is still TODO, patches are welcome.

Development

NOTE: Make sure you are in the Searx-Qt source root (where utils/, locales/, searxqt/ etc.. are).

NOTE: To run Searx-Qt without need to install:

# Copy the executable from ./bin to cwd (searx-qt source root)
cp bin/searx-qt ./

# Start
./searx-qt

Themes

Create new theme

A theme consists of icons, application css and search result/fail css.

Simple example to create a new theme from the default theme:

NOTE: Replace your_theme with the name of your new theme.

NOTE: For this example you should know basic CSS.

Setup structure for new theme.

cp -r ./themes/default/ ./themes/your_theme/

Edit the application style ./themes/your_theme/style.css.
Edit the css used for failed search result message ./themes/your_theme/html_fail.css.
Edit the css used for search results ./themes/your_theme/html_results.css
Edit the icons (don't change their size) ./themes/your_theme/icons/*.png
Open ./themes/your_theme/manifest.json and change the name variable to the pretty name of your new theme.
See if your theme is listed, when not there is a error:
```
python ./utils/themes_tool.py list
```

Compile the theme:

python ./utils/themes_tool.py make your_theme

Open Searx-Qt, go to settings, change to your new theme (it should be listed, else there is an error) and test it:
```
./searx-qt
```
Done? :-)

Translations

Searx-Qt will try to find a translation for your system locale and use that when found.

To test translations the system locale should be installed for that language, it doesn't have to be set for testing since we can easly override the LANG environment variable before executing Searx-Qt.

NOTE: The examples below are for a Dutch language translation, you should replace nl_NL with the i18n locale ID of the language you whish to translate.

Create new translation

Setup structure for new language:

# Update the searx-qt.pot template file.
./utils/locale_tool.sh -c

# Create directory structure.
mkdir -p ./locales/nl_NL/LC_MESSAGES/

# Copy the template file to our new directory.
cp ./locales/searx-qt.pot ./locales/nl_NL/LC_MESSAGES/searx-qt.po

# Check if our new language is found. It should be listed.
./utils/locale_tool.sh --list

Start working on the translation.

You can edit the ./locales/nl_NL/LC_MESSAGES/searx-qt.po file with a text editor or a special editor for translations (that can handle .po files like for example Poedit).
Compile the translation:
```
./utils/locale_tool.sh -m nl_NL
```

Test the translation:

# Note: overriding XDG_DATA_HOME only for debugging translations! Themes
#       will not work.
XDG_DATA_HOME="$(pwd -P)/" LC_ALL=nl_NL.UTF-8 ./searx-qt

Update existing translation

Update files:

# Update the .pot template file.
./utils/locale_tool.sh -c

# Update the translation it's .po file
./utils/locale_tool.sh -u nl_NL

Edit the translation:

You can edit the ./locales/nl_NL/LC_MESSAGES/searx-qt.po file with a text editor or a special editor for translations (that can handle .po files like for example Poedit).
Compile the translation:
```
./utils/locale_tool.sh -m nl_NL
```

Test the translation:

# Note: overriding XDG_DATA_HOME only for debugging translations! Themes
#       will not work.
XDG_DATA_HOME="$(pwd -P)/" LC_ALL=nl_NL.UTF-8 ./searx-qt

Searx-Qt

Desktop client application for Searx/SearXNG/searx-space

Index

Searx

SearXNG

searx-space

Debian / Ubuntu based

Arch based

Debian based

Arch based

Creating new profile

Deleting a profile

Theme

CLI output level

Verify (SSL)

Timeout

Proxy

User-agents

Extra headers

Parse HTML

Safe Search

Rules

Log

With Stats2 profile type

With User profile type

Instances table

Filter instances

Network

Require ASN privacy

Require IPv6

Version

Blacklist

Whitelist

Timeout

Search bar

Fallback

Random every

Random search button

Reload button

Search button

Search query input

Search options

Categories

Engines

Period

Language

Search results

Find text in results

Create new theme

Create new translation

Update existing translation