meilisearch @ v1.41.0

✨ Enhancement

• Support dynamic search rules with pinning by @YoEight in https://github.com/meilisearch/meilisearch/pull/6182 Introduce the Dynamic Search Rules (DSR) experimental feature, making it easy to promote the right content for the right searches with flexible, condition-based pinning. In this first version, rules can be triggered by query-based conditions such as empty queries or literal substring matches, as well as time windows. Multiple documents can be pinned in a defined order, and pins still work with filtering, pagination, facet distribution, hybrid search, and federated search. This feature is behind the dynamicSearchRules experimental feature flag.

  Rules can be created or updated with PATCH /dynamic-search-rules/{uid} and removed with DELETE /dynamic-search-rules/{uid}. In this first version, a rule can define query- or time-based conditions and pin specific documents at fixed positions in the results list.

    // PATCH /dynamic-search-rules/featured-wireless-headphones
    {
      "description": "Promote featured products for wireless headphone searches",
      "active": true,
      "conditions": [
        { "scope": "query", "contains": "headphone" }
      ],
      "actions": [
        {
          "selector": { "indexUid": "products", "id": "featured-headphones-001" },
          "action": { "type": "pin", "position": 0 }
        },
        {
          "selector": { "indexUid": "products", "id": "featured-headphones-002" },
          "action": { "type": "pin", "position": 1 }
        }
      ]
    }
  

• Default to useNetwork: true in sharded instances by @dureuill in https://github.com/meilisearch/meilisearch/pull/6278

  This release contains breaking changes for users of the network experimental feature.

  Search requests now use the network by default when (replicated) sharding is enabled

  § Breaking changes

  When network.leader is set in the instance, useNetwork now defaults to true in search requests when omitted.

  So, when a network is configured, the following:

  // POST /indexes/movies/search
  {
    "q": "Batman dark knight returns 1",
    "filter": "genres IN [Action, Adventure]",
    "facets": ["genres"],
    "limit": 5
    // No `useNetwork` is passed
  }
  

  is now equivalent to a federated search over multiples remotes so that all shards in the network are covered exactly once.

  // POST /multi-search
  {
      "federation": {
          "limit": 5,
          "facetsByIndex": {
              "movies": [
                  "genres"
              ]
          },
          "merge": {}
      },
      // assuming 3 remotes 0, 1, 2 and 3 shards a, b, c such that
      // 0 owns a, b
      // 1 owns b, c
      // 2 owns c, a
      "queries": [
          {
              "indexUid": "movies",
              "federationOptions": {
                  "remote": "0"
              },
              "q": "Batman dark knight returns 1",
              "filter": ["genres IN [Action, Adventure]", "_shard = a"]
          },
          {
              "indexUid": "movies",
              "federationOptions": {
                  "remote": "0" // useNetwork picked remote 0 for both shard a and b
              },
              "q": "Batman dark knight returns 1",
  		     "filter": ["genres IN [Action, Adventure]", "_shard = b"]
          },
          {
              "indexUid": "movies",
              "federationOptions": {
                  "remote": "2"
              },
              "q": "Batman dark knight returns 1",
  		     "filter": ["genres IN [Action, Adventure]", "_shard = c"]
          }
      ]
  }
  

  Responding with documents from all the shards:

  {
    "hits": [
      {
        "id": 123025,
        "title": "Batman: The Dark Knight Returns, Part 1",
        "overview": "Batman has not been seen for ten years. A new breed of criminal ravages Gotham City, forcing 55-year-old Bruce Wayne back into the cape and cowl. But, does he still have what it takes to fight crime in a new era?",
        "genres": [
          "Action",
          "Animation",
          "Mystery"
        ],
        "poster": "https://image.tmdb.org/t/p/w500/kkjTbwV1Xnj8wBL52PjOcXzTbnb.jpg",
        "release_date": 1345507200,
        "_federation": {
          "indexUid": "mieli",
          "queriesPosition": 2,
          "weightedRankingScore": 0.9894586894586894,
          "remote": "2"
        }
      },
      {
        "id": 142061,
        "title": "Batman: The Dark Knight Returns, Part 2",
        "overview": "Batman has stopped the reign of terror that The Mutants had cast upon his city.  Now an old foe wants a reunion and the government wants The Man of Steel to put a stop to Batman.",
        "genres": [
          "Action",
          "Animation",
          "Mystery"
        ],
        "poster": "https://image.tmdb.org/t/p/w500/arEZYd6uMOFTILne9Ux0A8qctMe.jpg",
        "release_date": 1357171200,
        "_federation": {
          "indexUid": "mieli",
          "queriesPosition": 1,
          "weightedRankingScore": 0.9894558963186414,
          "remote": "0"
        }
      },
      {
        "id": 16234,
        "title": "Batman Beyond: Return of the Joker",
        "overview": "The Joker is back with a vengeance, and Gotham's newest Dark Knight needs answers as he stands alone to face Gotham's most infamous Clown Prince of Crime.",
        "genres": [
          "Animation",
          "Family",
          "Action",
          "Science Fiction"
        ],
        "poster": "https://image.tmdb.org/t/p/w500/7RlBs0An83fqAuKfwH5gKMcqgMc.jpg",
        "release_date": 976579200,
        "_federation": {
          "indexUid": "mieli",
          "queriesPosition": 1,
          "weightedRankingScore": 0.9427964918160996,
          "remote": "0"
        }
      },
      {
        "id": 155,
        "title": "The Dark Knight",
        "overview": "Batman raises the stakes in his war on crime. With the help of Lt. Jim Gordon and District Attorney Harvey Dent, Batman sets out to dismantle the remaining criminal organizations that plague the streets. The partnership proves to be effective, but they soon find themselves prey to a reign of chaos unleashed by a rising criminal mastermind known to the terrified citizens of Gotham as the Joker.",
        "genres": [
          "Drama",
          "Action",
          "Crime",
          "Thriller"
        ],
        "poster": "https://image.tmdb.org/t/p/w500/qJ2tW6WMUDux911r6m7haRef0WH.jpg",
        "release_date": 1216166400,
        "_federation": {
          "indexUid": "mieli",
          "queriesPosition": 1,
          "weightedRankingScore": 0.5784178187403994,
          "remote": "0"
        }
      },
      {
        "id": 49026,
        "title": "The Dark Knight Rises",
        "overview": "Following the death of District Attorney Harvey Dent, Batman assumes responsibility for Dent's crimes to protect the late attorney's reputation and is subsequently hunted by the Gotham City Police Department. Eight years later, Batman encounters the mysterious Selina Kyle and the villainous Bane, a new terrorist leader who overwhelms Gotham's finest. The Dark Knight resurfaces to protect a city that has branded him an enemy.",
        "genres": [
          "Action",
          "Crime",
          "Drama",
          "Thriller"
        ],
        "poster": "https://image.tmdb.org/t/p/w500/vzvKcPQ4o7TjWeGIn0aGC9FeVNu.jpg",
        "release_date": 1342396800,
        "_federation": {
          "indexUid": "mieli",
          "queriesPosition": 2,
          "weightedRankingScore": 0.5772657450076805,
          "remote": "2"
        }
      }
    ],
    "query": "Batman dark knight returns 1",
    "processingTimeMs": 173,
    "limit": 5,
    "offset": 0,
    "estimatedTotalHits": 47,
    "facetDistribution": {
      "genres": {
        "Action": 46,
        "Adventure": 15,
        "Animation": 34,
        "Comedy": 3,
        "Crime": 14,
        "Drama": 6,
        "Family": 15,
        "Fantasy": 8,
        "Horror": 1,
        "Mystery": 4,
        "Romance": 1,
        "Science Fiction": 14,
        "TV Movie": 4,
        "Thriller": 4,
        "Western": 1
      }
    },
    "facetStats": {},
    "requestUid": "019bbcf4-a609-7701-8d82-d370611adfb3",
    "remoteErrors": {}
  }
  

  This change allows to naturally query all the documents in a sharded context. When network.leader is not present (in particular, when no network was defined), then the behavior is identical to previous versions.

• useNetwork picks local instance if it owns the shard by @dureuill in https://github.com/meilisearch/meilisearch/pull/6287 To prevent unnecessary network activity, when deciding which remote to ask for a shard in a search over the network, Meilisearch will now always pick the local instance if it owns the shard.

• More efficient FST building by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/6279 It improves the construction of the word FST (word dictionary) by removing the need for a full scan of the word docids database. This should drastically improve the database's performance when inserting a large number of documents, even when inserting only a few.

🪲 Bug fixes

• Handle empty VERSION file as missing by @themavik in https://github.com/meilisearch/meilisearch/pull/6275

🔩 Miscellaneous

• Remap post processing targets for better benchmark readability by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/6285
• Update the description of the get task's document payload route by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/6284

New Contributors

• @themavik made their first contribution in https://github.com/meilisearch/meilisearch/pull/6275