Meilisearch v1.2 introduces new filter operators IS NULL and IS EMPTY, a new route to delete documents with filters, and significant relevancy and performance updates.

🧰 All official Meilisearch integrations (including SDKs, clients, and other tools) are compatible with this Meilisearch release. Integration deployment happens between 4 to 48 hours after a new version becomes available.

Some SDKs might not include all new features—consult the project repository for detailed information. Is a feature you need missing from your chosen SDK? Create an issue letting us know you need it, or, for open-source karma points, open a PR implementing it (we'll love you for that ❤️).

New features and improvements 🔥

Delete documents by filter

Meilisearch v1.2 allows you to use filters to delete documents with the new /documents/delete route:

curl -X POST http://localhost:7700/indexes/dogs/documents/delete \
  -H 'Content-Type: application/json' \
  --data-binary '{ "filter": ["doggo = 'bernese mountain'", "face = cute"] }'

Fields must be set as filterable before you can use them as filters.

Meilisearch will return a summarized task object:

{
  "taskUid": 242,
  "indexUid": "dogs",
  "status": "enqueued",
  "type": "documentDeletion",
  "enqueuedAt": "2023-05-03T11:01:58.721841Z"
}

Use the returned taskUid with the task route to check the task status.

Done by @dureuill and @irevoire in #3550.

Get documents by filter

You can now use filters in the GET endpoint of the /documents route:

curl -X GET 'http://localhost:7700/indexes/dogs/documents?limit=1&filter=doggo=bernese'

You can also use the new /documents/fetch route to handle complex filters:

curl -X POST http://localhost:7700/indexes/dogs/documents/fetch \
  -H 'Content-Type: application/json' \
  --data-binary '{ "limit": 1, "filter": "doggo = bernese" }'

/documents/fetch accepts following parameters: limit, offset, fields, and filter.

Fields must be set as filterable before you can use them as filters.

Done by @dureuill and @irevoire in #3570.

New filter operators: IS EMPTY and IS NULL

This release introduces two new filter operators. IS EMPTY matches existing fields with a valid, but empty value. IS NULL matches existing fields with an explicit null value.

Given the following documents:

[{
    "id": 0,
    "color": []
},
{
    "id": 1,
    "color": null
},
{
    "id": 2,
},
]

color IS EMPTY matches document 0.

color IS NULL matches document 1.

Both operators can be used together with the NOT operator: color IS NOT EMPTY and NOT color IS EMPTY match document 1. color IS NOT NULL and NOT color IS NULL match document 0.

Neither operator will match documents missing the specified field.

Done by @Kerollmops in #3571.

Improve relevancy and search performance

The search engine has seen significant refactoring. This brings performance and relevancy improvements, as well as setting the foundations for new features related to search customization. Stay tuned!

If you have any questions or experience unexpected behavior, feel free to reach out.

Done by @loiclec, @dureuill, @ManyTheFish, @Kerollmops and @irevoire in #3542.

Performance improvements

• The fastest 75 percent of queries now consistently answer below 50ms on our test setup
• Complexity of search queries has been limited to limit CPU time and RAM consumption. More specifically:
  • A single term is now limited to 150 possible typo matches for queries containing 1 typo, and to 50 possibilities for queries containing 2 typos
  • Both single word or multi word queries consider a maximum of 50 synonyms
  • The total number of words for all synonyms of a single term cannot exceed 100
  • Queries can now contain a maximum of 1000 words. This change only impacts queries containing very long phrases. Search terms (words not inside a phrase, and individual phrases) remain limited to 10 per query
• Geo search improvements:
  • Increased performance when geosorting small sets of documents
  • Descending sort is now as performant as ascending sort
• Address assorted minor performance issues: https://github.com/meilisearch/meilisearch/issues/3124, https://github.com/meilisearch/meilisearch/issues/3378, https://github.com/meilisearch/meilisearch/issues/2202, https://github.com/meilisearch/meilisearch/issues/3123

Relevancy improvements

• The exactness ranking rule no longer treats synonyms as exact matches. This boosts documents containing the query exactly as typed by the user. Solves https://github.com/meilisearch/meilisearch/issues/3410
• Meilisearch will always sort results as if the words ranking rule were present with a higher priority than the attributes, exactness, typo and proximity ranking rules. This happens even if the words ranking rule has been removed or set with a lower priority. Search behavior when using the specified ranking rules before or without words was confusing and hurt relevancy
• Split words are now also treated as possible digrams. This means the query whit ehorse may match documents containing white horse, which improves the relevancy of typo tolerance
• N-grams and split words are now ranked lower in comparison to the exact word when processing the typo ranking rule. For example, a query for sun flower will now return a document titled Sun Flower before one titled Sunflower
• Ranking rule behavior no longer depends on the number of ranked documents, improving consistency in terms of both relevancy and latency. Solves https://github.com/meilisearch/meilisearch/issues/3356

Automated task deletion

Maximum number of tasks the task queue can hold now limited to 1M. Fixes issues with enqueuing new tasks when the database is full.

Once the task queue reaches this limit, Meilisearch will try to delete the oldest 100k tasks. If not all 100k tasks can be deleted due to some of them being unfinished, Meilisearch will delete as many tasks as possible. If there are no unfinished tasks, Meilisearch will not delete anything but still enqueue new tasks as usual.

The engine will stop you from adding new tasks once the hard limit of 10GiB of tasks is reached (that’s between 5M and 15M of tasks depending on your workflow). In this case, Meilisearch will try to automatically delete unfinished tasks. If this is unsuccessful, it will not enqueue new tasks, and log a warning notifying the user the engine is not working properly.

Done by @irevoire in #3693.

Language support improvements

• Split camelCase in Latin segmenter
• Improve Arabic: normalizer and segmentation

Done by @ManyTheFish, @akeamc, @DrAliRagab, @goodhoko, and @mosuka in #3702 and in Charabia v0.7.2

Other improvements

• Introduce experimental feature to reduce the RAM usage. Can be useful if Meilisearch is sometimes killed by your OS. Check out this discussion to get more information and how to use it (#3651) @Kerollmops
• Add boolean support for CSV documents (#3576) @irevoire
• Improve geosort errors during sorting and filtering (#3638 and #3631) @cymruu
• Improve error message when payload is too large (#3739) @cymruu

Fixes 🐞

• Improve the GET /health route by ensuring the internal DB is not down (#3647) @irevoire
• Handle single quotation mark (’) and apostrophe (') the same way (#3702) @ManyTheFish
• Fix the sort error message: regression seen in v1.1 (#3749 and #3755) @ManyTheFish

Misc

• Dependencies upgrade
  • Bump the Meilisearch dependencies (#3661) @Kerollmops
  • Security: bump h2 from 0.3.15 to 0.3.17 (#3674)
  • Update clippy toolchain from v1.67 to v1.69 (#3695) @curquiza
  • Bump GitHub actions dependencies: #3554, #3551, #3552, #3553, #3633
• CIs and tests
  • Add scheduled tests for all features to Actions (#3510) @jlucktay
  • Clean CI file names (#3567) @curquiza
  • Enable cache again in test suite CI (#3587) @curquiza
  • Remove CLI changes for clippy (#3464) @dureuill
  • Improve docker cache in CIs (#3566) @inductor
  • Remove Uffizzi because the team no longer uses it (#3694) @curquiza
  • Add SDK tests to CI (#3709) @curquiza
  • Use URL of our self-hosted bors instance (#3721) @curquiza
  • Improve index-scheduler tests (#3639) @Sufflope
• Documentation
  • Add newline after the meilisearch version in the issue template (#3636) @bidoubiwa
  • Add sprint issue to template issues (#3643) @curquiza
  • Update the prototype section in CONTRIBUTING.md (#3649) @curquiza
  • Update README to reference new docs website (#3666, #3710, #3718 and #3720) @guimachiavelli @roy9495 @Kerollmops @curquiza
  • Move comments above keys in config.toml (#3731) @jirutka
• Misc
  • Milli scope: when updating settings, check if primary key actually changes before throwing an error (#3608) @GregoryConrad
  • Remove unused snapshot files (#3696) @irevoire
  • Add missing analytics (#3738) @irevoire

❤️ Thanks again to our external contributors:

• Meilisearch: @cymruu, @GregoryConrad, @inductor, @jirutka, @jlucktay, @roy9495, and @Sufflope.
• Charabia: @akeamc, @DrAliRagab, @goodhoko, and @mosuka.