Comment by mwambua

Comment by mwambua 3 days ago

3 replies

Tangential question. What do people use for search? What search engines provide the best quality to cost ratios?

Also are there good solutions for searching through a local collection of documents?

andai 3 days ago

ddg (python lib) is free and I'd say good enough for most tasks. (I think the endpoint is unofficial, but from what I've heard it's fine for typical usage.)

There's also google, which gives you 100 requests a day or something.

Here's the search.py I use

    import os
    import json
    from req import get

    # https://programmablesearchengine.google.com/controlpanel/create
    GOOGLE_SEARCH_API_KEY = os.getenv('GOOGLE_SEARCH_API_KEY')
    GOOGLE_SEARCH_API_ID = os.getenv('GOOGLE_SEARCH_API_ID')

    url = "https://customsearch.googleapis.com/customsearch/v1"

    def search(query):
        data = {
            "q": query,
            "cx": GOOGLE_SEARCH_API_ID,
            "key": GOOGLE_SEARCH_API_KEY,
        }
        results_json = get(url, data)
        results = json.loads(results_json)
        results = results["items"]
        return results

    if __name__ == "__main__":
        while True:
            query = input('query: ')
            results = search(query)
            print(results)

and the ddg version

    from duckduckgo_search import DDGS

    def search(query, max_results=8):
        results = DDGS().text(query, max_results=max_results)
        return results
  • mwambua 3 days ago

    Oh, nice! Thanks! This reminds me of the unofficial yahoo finance api.

nickthegreek 2 days ago

just setup searxng yesterday and mcp for it in lm studio to be able to search the net for answers to simple queries. small ibm granite worked surpassingly well, while oss20b seemed to be looping searches.