no-sql-engines.html 22 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266
  1. <!DOCTYPE html>
  2. <html lang="en">
  3. <head>
  4. <meta charset="utf-8" />
  5. <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  6. <meta name="viewport" content="width=device-width, initial-scale=1">
  7. <title>Query SQL and NoSQL servers &#8212; Searx Documentation (Searx-1.1.0.tex)</title>
  8. <link rel="stylesheet" type="text/css" href="../_static/pygments.css" />
  9. <link rel="stylesheet" type="text/css" href="../_static/searx.css" />
  10. <link rel="stylesheet" type="text/css" href="../_static/tabs.css" />
  11. <script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
  12. <script src="../_static/jquery.js"></script>
  13. <script src="../_static/underscore.js"></script>
  14. <script src="../_static/_sphinx_javascript_frameworks_compat.js"></script>
  15. <script src="../_static/doctools.js"></script>
  16. <script src="../_static/sphinx_highlight.js"></script>
  17. <link rel="index" title="Index" href="../genindex.html" />
  18. <link rel="search" title="Search" href="../search.html" />
  19. <link rel="next" title="Plugins builtin" href="plugins.html" />
  20. <link rel="prev" title="Search in indexers" href="indexer-engines.html" />
  21. </head><body>
  22. <div class="related" role="navigation" aria-label="related navigation">
  23. <h3>Navigation</h3>
  24. <ul>
  25. <li class="right" style="margin-right: 10px">
  26. <a href="../genindex.html" title="General Index"
  27. accesskey="I">index</a></li>
  28. <li class="right" >
  29. <a href="../py-modindex.html" title="Python Module Index"
  30. >modules</a> |</li>
  31. <li class="right" >
  32. <a href="plugins.html" title="Plugins builtin"
  33. accesskey="N">next</a> |</li>
  34. <li class="right" >
  35. <a href="indexer-engines.html" title="Search in indexers"
  36. accesskey="P">previous</a> |</li>
  37. <li class="nav-item nav-item-0"><a href="../index.html">Searx Documentation (Searx-1.1.0.tex)</a> &#187;</li>
  38. <li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Administrator documentation</a> &#187;</li>
  39. <li class="nav-item nav-item-this"><a href="">Query SQL and NoSQL servers</a></li>
  40. </ul>
  41. </div>
  42. <div class="document">
  43. <div class="documentwrapper">
  44. <div class="bodywrapper">
  45. <div class="body" role="main">
  46. <section id="query-sql-and-nosql-servers">
  47. <h1>Query SQL and NoSQL servers<a class="headerlink" href="#query-sql-and-nosql-servers" title="Permalink to this heading">¶</a></h1>
  48. <section id="sql">
  49. <h2>SQL<a class="headerlink" href="#sql" title="Permalink to this heading">¶</a></h2>
  50. <p>SQL servers are traditional databases with predefined data schema. Furthermore,
  51. modern versions also support BLOB data.</p>
  52. <p>You can search in the following servers:</p>
  53. <ul class="simple">
  54. <li><p><a class="reference internal" href="#postgresql">PostgreSQL</a></p></li>
  55. <li><p><a class="reference internal" href="#mysql">MySQL</a></p></li>
  56. <li><p><a class="reference internal" href="#sqlite">SQLite</a></p></li>
  57. </ul>
  58. <p>The configuration of the new database engines are similar. You must put a valid
  59. SELECT SQL query in <code class="docutils literal notranslate"><span class="pre">query_str</span></code>. At the moment you can only bind at most
  60. one parameter in your query.</p>
  61. <p>Do not include LIMIT or OFFSET in your SQL query as the engines
  62. rely on these keywords during paging.</p>
  63. <section id="postgresql">
  64. <h3>PostgreSQL<a class="headerlink" href="#postgresql" title="Permalink to this heading">¶</a></h3>
  65. <p>Required PyPi package: <code class="docutils literal notranslate"><span class="pre">psychopg2</span></code></p>
  66. <p>You can find an example configuration below:</p>
  67. <div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">postgresql</span><span class="w"></span>
  68. <span class="w"> </span><span class="nt">engine </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">postgresql</span><span class="w"></span>
  69. <span class="w"> </span><span class="nt">database </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">my_database</span><span class="w"></span>
  70. <span class="w"> </span><span class="nt">username </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">searx</span><span class="w"></span>
  71. <span class="w"> </span><span class="nt">password </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">password</span><span class="w"></span>
  72. <span class="w"> </span><span class="nt">query_str </span><span class="p">:</span><span class="w"> </span><span class="s">&#39;SELECT</span><span class="nv"> </span><span class="s">*</span><span class="nv"> </span><span class="s">from</span><span class="nv"> </span><span class="s">my_table</span><span class="nv"> </span><span class="s">WHERE</span><span class="nv"> </span><span class="s">my_column</span><span class="nv"> </span><span class="s">=</span><span class="nv"> </span><span class="s">%(query)s&#39;</span><span class="w"></span>
  73. <span class="w"> </span><span class="nt">shortcut </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">psql</span><span class="w"></span>
  74. </pre></div>
  75. </div>
  76. <section id="available-options">
  77. <h4>Available options<a class="headerlink" href="#available-options" title="Permalink to this heading">¶</a></h4>
  78. <ul class="simple">
  79. <li><p><code class="docutils literal notranslate"><span class="pre">host</span></code>: IP address of the host running PostgreSQL. By default it is <code class="docutils literal notranslate"><span class="pre">127.0.0.1</span></code>.</p></li>
  80. <li><p><code class="docutils literal notranslate"><span class="pre">port</span></code>: Port number PostgreSQL is listening on. By default it is <code class="docutils literal notranslate"><span class="pre">5432</span></code>.</p></li>
  81. <li><p><code class="docutils literal notranslate"><span class="pre">database</span></code>: Name of the database you are connecting to.</p></li>
  82. <li><p><code class="docutils literal notranslate"><span class="pre">username</span></code>: Name of the user connecting to the database.</p></li>
  83. <li><p><code class="docutils literal notranslate"><span class="pre">password</span></code>: Password of the database user.</p></li>
  84. <li><p><code class="docutils literal notranslate"><span class="pre">query_str</span></code>: Query string to run. Keywords like <code class="docutils literal notranslate"><span class="pre">LIMIT</span></code> and <code class="docutils literal notranslate"><span class="pre">OFFSET</span></code> are not allowed. Required.</p></li>
  85. <li><p><code class="docutils literal notranslate"><span class="pre">limit</span></code>: Number of returned results per page. By default it is 10.</p></li>
  86. </ul>
  87. </section>
  88. </section>
  89. <section id="mysql">
  90. <h3>MySQL<a class="headerlink" href="#mysql" title="Permalink to this heading">¶</a></h3>
  91. <p>Required PyPi package: <code class="docutils literal notranslate"><span class="pre">mysql-connector-python</span></code></p>
  92. <p>This is an example configuration for quering a MySQL server:</p>
  93. <div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">mysql</span><span class="w"></span>
  94. <span class="w"> </span><span class="nt">engine </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">mysql_server</span><span class="w"></span>
  95. <span class="w"> </span><span class="nt">database </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">my_database</span><span class="w"></span>
  96. <span class="w"> </span><span class="nt">username </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">searx</span><span class="w"></span>
  97. <span class="w"> </span><span class="nt">password </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">password</span><span class="w"></span>
  98. <span class="w"> </span><span class="nt">limit </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">5</span><span class="w"></span>
  99. <span class="w"> </span><span class="nt">query_str </span><span class="p">:</span><span class="w"> </span><span class="s">&#39;SELECT</span><span class="nv"> </span><span class="s">*</span><span class="nv"> </span><span class="s">from</span><span class="nv"> </span><span class="s">my_table</span><span class="nv"> </span><span class="s">WHERE</span><span class="nv"> </span><span class="s">my_column=%(query)s&#39;</span><span class="w"></span>
  100. <span class="w"> </span><span class="nt">shortcut </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">mysql</span><span class="w"></span>
  101. </pre></div>
  102. </div>
  103. <section id="id1">
  104. <h4>Available options<a class="headerlink" href="#id1" title="Permalink to this heading">¶</a></h4>
  105. <ul class="simple">
  106. <li><p><code class="docutils literal notranslate"><span class="pre">host</span></code>: IP address of the host running MySQL. By default it is <code class="docutils literal notranslate"><span class="pre">127.0.0.1</span></code>.</p></li>
  107. <li><p><code class="docutils literal notranslate"><span class="pre">port</span></code>: Port number MySQL is listening on. By default it is <code class="docutils literal notranslate"><span class="pre">3306</span></code>.</p></li>
  108. <li><p><code class="docutils literal notranslate"><span class="pre">database</span></code>: Name of the database you are connecting to.</p></li>
  109. <li><p><code class="docutils literal notranslate"><span class="pre">auth_plugin</span></code>: Authentication plugin to use. By default it is <code class="docutils literal notranslate"><span class="pre">caching_sha2_password</span></code>.</p></li>
  110. <li><p><code class="docutils literal notranslate"><span class="pre">username</span></code>: Name of the user connecting to the database.</p></li>
  111. <li><p><code class="docutils literal notranslate"><span class="pre">password</span></code>: Password of the database user.</p></li>
  112. <li><p><code class="docutils literal notranslate"><span class="pre">query_str</span></code>: Query string to run. Keywords like <code class="docutils literal notranslate"><span class="pre">LIMIT</span></code> and <code class="docutils literal notranslate"><span class="pre">OFFSET</span></code> are not allowed. Required.</p></li>
  113. <li><p><code class="docutils literal notranslate"><span class="pre">limit</span></code>: Number of returned results per page. By default it is 10.</p></li>
  114. </ul>
  115. </section>
  116. </section>
  117. <section id="sqlite">
  118. <h3>SQLite<a class="headerlink" href="#sqlite" title="Permalink to this heading">¶</a></h3>
  119. <p>You can read from your database <code class="docutils literal notranslate"><span class="pre">my_database</span></code> using this example configuration:</p>
  120. <div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">sqlite</span><span class="w"></span>
  121. <span class="w"> </span><span class="nt">engine </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">sqlite</span><span class="w"></span>
  122. <span class="w"> </span><span class="nt">shortcut</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">sq</span><span class="w"></span>
  123. <span class="w"> </span><span class="nt">database </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">my_database</span><span class="w"></span>
  124. <span class="w"> </span><span class="nt">query_str </span><span class="p">:</span><span class="w"> </span><span class="s">&#39;SELECT</span><span class="nv"> </span><span class="s">*</span><span class="nv"> </span><span class="s">FROM</span><span class="nv"> </span><span class="s">my_table</span><span class="nv"> </span><span class="s">WHERE</span><span class="nv"> </span><span class="s">my_column=:query&#39;</span><span class="w"></span>
  125. </pre></div>
  126. </div>
  127. <section id="id2">
  128. <h4>Available options<a class="headerlink" href="#id2" title="Permalink to this heading">¶</a></h4>
  129. <ul class="simple">
  130. <li><p><code class="docutils literal notranslate"><span class="pre">database</span></code>: Name of the database you are connecting to.</p></li>
  131. <li><p><code class="docutils literal notranslate"><span class="pre">query_str</span></code>: Query string to run. Keywords like <code class="docutils literal notranslate"><span class="pre">LIMIT</span></code> and <code class="docutils literal notranslate"><span class="pre">OFFSET</span></code> are not allowed. Required.</p></li>
  132. <li><p><code class="docutils literal notranslate"><span class="pre">limit</span></code>: Number of returned results per page. By default it is 10.</p></li>
  133. </ul>
  134. </section>
  135. </section>
  136. </section>
  137. <section id="nosql">
  138. <h2>NoSQL<a class="headerlink" href="#nosql" title="Permalink to this heading">¶</a></h2>
  139. <p>NoSQL data stores are used for storing arbitrary data without first defining their
  140. structure. To query the supported servers, you must install their drivers using PyPi.</p>
  141. <p>You can search in the following servers:</p>
  142. <ul class="simple">
  143. <li><p><a class="reference internal" href="#redis">Redis</a></p></li>
  144. <li><p><a class="reference internal" href="#mongodb">MongoDB</a></p></li>
  145. </ul>
  146. <section id="redis">
  147. <h3>Redis<a class="headerlink" href="#redis" title="Permalink to this heading">¶</a></h3>
  148. <p>Reqired PyPi package: <code class="docutils literal notranslate"><span class="pre">redis</span></code></p>
  149. <p>Example configuration:</p>
  150. <div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">mystore</span><span class="w"></span>
  151. <span class="w"> </span><span class="nt">engine </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">redis_server</span><span class="w"></span>
  152. <span class="w"> </span><span class="nt">exact_match_only </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">True</span><span class="w"></span>
  153. <span class="w"> </span><span class="nt">host </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">127.0.0.1</span><span class="w"></span>
  154. <span class="w"> </span><span class="nt">port </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">6379</span><span class="w"></span>
  155. <span class="w"> </span><span class="nt">password </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">secret-password</span><span class="w"></span>
  156. <span class="w"> </span><span class="nt">db </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">0</span><span class="w"></span>
  157. <span class="w"> </span><span class="nt">shortcut </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">rds</span><span class="w"></span>
  158. <span class="w"> </span><span class="nt">enable_http </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">True</span><span class="w"></span>
  159. </pre></div>
  160. </div>
  161. <section id="id3">
  162. <h4>Available options<a class="headerlink" href="#id3" title="Permalink to this heading">¶</a></h4>
  163. <ul class="simple">
  164. <li><p><code class="docutils literal notranslate"><span class="pre">host</span></code>: IP address of the host running Redis. By default it is <code class="docutils literal notranslate"><span class="pre">127.0.0.1</span></code>.</p></li>
  165. <li><p><code class="docutils literal notranslate"><span class="pre">port</span></code>: Port number Redis is listening on. By default it is <code class="docutils literal notranslate"><span class="pre">6379</span></code>.</p></li>
  166. <li><p><code class="docutils literal notranslate"><span class="pre">password</span></code>: Password if required by Redis.</p></li>
  167. <li><p><code class="docutils literal notranslate"><span class="pre">db</span></code>: Number of the database you are connecting to.</p></li>
  168. <li><p><code class="docutils literal notranslate"><span class="pre">exact_match_only</span></code>: Enable if you need exact matching. By default it is <code class="docutils literal notranslate"><span class="pre">True</span></code>.</p></li>
  169. </ul>
  170. </section>
  171. </section>
  172. <section id="mongodb">
  173. <h3>MongoDB<a class="headerlink" href="#mongodb" title="Permalink to this heading">¶</a></h3>
  174. <p>Required PyPi package: <code class="docutils literal notranslate"><span class="pre">pymongo</span></code></p>
  175. <p>Below is an example configuration for using a MongoDB collection:</p>
  176. <div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">mymongo</span><span class="w"></span>
  177. <span class="w"> </span><span class="nt">engine </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">mongodb</span><span class="w"></span>
  178. <span class="w"> </span><span class="nt">shortcut </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">icm</span><span class="w"></span>
  179. <span class="w"> </span><span class="nt">host </span><span class="p">:</span><span class="w"> </span><span class="s">&#39;127.0.0.1&#39;</span><span class="w"></span>
  180. <span class="w"> </span><span class="nt">port </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">27017</span><span class="w"></span>
  181. <span class="w"> </span><span class="nt">database </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">personal</span><span class="w"></span>
  182. <span class="w"> </span><span class="nt">collection </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">income</span><span class="w"></span>
  183. <span class="w"> </span><span class="nt">key </span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">month</span><span class="w"></span>
  184. <span class="w"> </span><span class="nt">enable_http</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">True</span><span class="w"></span>
  185. </pre></div>
  186. </div>
  187. <section id="id4">
  188. <h4>Available options<a class="headerlink" href="#id4" title="Permalink to this heading">¶</a></h4>
  189. <ul class="simple">
  190. <li><p><code class="docutils literal notranslate"><span class="pre">host</span></code>: IP address of the host running MongoDB. By default it is <code class="docutils literal notranslate"><span class="pre">127.0.0.1</span></code>.</p></li>
  191. <li><p><code class="docutils literal notranslate"><span class="pre">port</span></code>: Port number MongoDB is listening on. By default it is <code class="docutils literal notranslate"><span class="pre">27017</span></code>.</p></li>
  192. <li><p><code class="docutils literal notranslate"><span class="pre">password</span></code>: Password if required by Redis.</p></li>
  193. <li><p><code class="docutils literal notranslate"><span class="pre">database</span></code>: Name of the database you are connecting to.</p></li>
  194. <li><p><code class="docutils literal notranslate"><span class="pre">collection</span></code>: Name of the collection you want to search in.</p></li>
  195. <li><p><code class="docutils literal notranslate"><span class="pre">exact_match_only</span></code>: Enable if you need exact matching. By default it is <code class="docutils literal notranslate"><span class="pre">True</span></code>.</p></li>
  196. </ul>
  197. </section>
  198. </section>
  199. </section>
  200. </section>
  201. <div class="clearer"></div>
  202. </div>
  203. </div>
  204. </div>
  205. <span id="sidebar-top"></span>
  206. <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
  207. <div class="sphinxsidebarwrapper">
  208. <p class="logo"><a href="../index.html">
  209. <img class="logo" src="../_static/searx_logo_small.png" alt="Logo"/>
  210. </a></p>
  211. <h3>Project Links</h3>
  212. <ul>
  213. <li><a href="https://searx.github.io/searx/blog/index.html">Blog</a>
  214. <li><a href="https://github.com/searx/searx">Source</a>
  215. <li><a href="https://github.com/searx/searx/wiki">Wiki</a>
  216. <li><a href="https://twitter.com/Searx_engine">Twitter</a>
  217. <li><a href="https://github.com/searx/searx/issues">Issue Tracker</a>
  218. </ul><h3>Navigation</h3>
  219. <ul>
  220. <li><a href="../index.html">Overview</a>
  221. <ul>
  222. <li><a href="index.html">Administrator documentation</a>
  223. <ul>
  224. <li>Previous: <a href="indexer-engines.html" title="previous chapter">Search in indexers</a>
  225. <li>Next: <a href="plugins.html" title="next chapter">Plugins builtin</a></ul>
  226. </li>
  227. </ul>
  228. </li>
  229. </ul>
  230. <div id="searchbox" style="display: none" role="search">
  231. <h3 id="searchlabel">Quick search</h3>
  232. <div class="searchformwrapper">
  233. <form class="search" action="../search.html" method="get">
  234. <input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
  235. <input type="submit" value="Go" />
  236. </form>
  237. </div>
  238. </div>
  239. <script>document.getElementById('searchbox').style.display = "block"</script>
  240. </div>
  241. </div>
  242. <div class="clearer"></div>
  243. </div>
  244. <div class="footer" role="contentinfo">
  245. &#169; Copyright 2015-2022, Adam Tauber, Noémi Ványi.
  246. Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 5.3.0.
  247. </div>
  248. <script src="../_static/version_warning_offset.js"></script>
  249. </body>
  250. </html>