Add /indexes/{indexUID}/fields route to get all field names
#5718
Conversation
This commit introduces a new API endpoint `/indexes/{indexUid}/fields` that allows users to fetch all field names in an index, including nested fields. It also includes corresponding tests to validate the functionality, ensuring that the endpoint returns the correct fields for both empty and populated indexes, as well as handling errors for non-existent indexes.
|
Hey @qdequele, After discussing with the team, we could accept a minimal implementation that doesn't contain the field Metadata.
The keys API use an autopagination helper, see below: meilisearch/crates/meilisearch/src/routes/api_key.rs Lines 189 to 193 in 7dfb207 |
33ae0b0 to
dfa6ae7
Compare
| use super::{Pagination, PAGINATION_DEFAULT_LIMIT}; | ||
|
|
||
| /// Field configuration for a specific field in the index | ||
| #[derive(Debug, Serialize, Clone, ToSchema)] |
There was a problem hiding this comment.
All the structures you added with ToSchema need to be added to the big list of structures in the main OpenApi structure.
|
|
||
| /// Field configuration for a specific field in the index | ||
| #[derive(Debug, Serialize, Clone, ToSchema)] | ||
| #[serde(rename_all = "camelCase")] |
There was a problem hiding this comment.
Also to be added on all other schema-deriving structures
| #[serde(rename_all = "camelCase")] | |
| #[serde(rename_all = "camelCase")] | |
| #[schema(rename_all = "camelCase")] |
| fn into_pagination(self) -> Pagination { | ||
| Pagination { offset: self.offset.0, limit: self.limit.0 } | ||
| } |
There was a problem hiding this comment.
No reason to take ownership here, it forces you to clone the fields later on
| fn into_pagination(self) -> Pagination { | |
| Pagination { offset: self.offset.0, limit: self.limit.0 } | |
| } | |
| fn to_pagination(&self) -> Pagination { | |
| Pagination { offset: *self.offset.0, limit: *self.limit.0 } | |
| } |
| fn parse_filter(expr: &str) -> Vec<FilterCond> { | ||
| let mut conditions = Vec::new(); | ||
| for cond_str in expr.split("&&") { | ||
| let cond_str = cond_str.trim(); | ||
| if cond_str.is_empty() { | ||
| continue; | ||
| } | ||
| if let Some((lhs, rhs)) = cond_str.split_once('=') { | ||
| let path: Vec<String> = lhs.trim().split('.').map(|s| s.trim().to_string()).collect(); | ||
| let value = matches!(rhs.trim(), "true" | "True" | "TRUE"); | ||
| conditions.push(FilterCond::BoolEq { path, value }); | ||
| } else if let Some((lhs, rhs)) = cond_str.split_once(':') { | ||
| let path: Vec<String> = lhs.trim().split('.').map(|s| s.trim().to_string()).collect(); | ||
| conditions.push(FilterCond::Contains { path, value: rhs.trim().to_string() }); | ||
| } | ||
| } | ||
| conditions | ||
| } |
There was a problem hiding this comment.
Can't fields contain ampersands, points or colons? If they can we need to provide a way to escape the path. I had this exact issue with embedder names last week and switching from a splitted iterator to a closure yielding the next potentially-escaped part was very easy
meilisearch/crates/meilisearch/src/routes/indexes/render.rs
Lines 400 to 417 in 6113831
| } | ||
|
|
||
| fn field_satisfies(field: &Field, conds: &[FilterCond]) -> bool { | ||
| let field_value = serde_json::to_value(field).unwrap_or_default(); |
There was a problem hiding this comment.
Not fond of having to serialize every field metadata. Something like that would be more efficient
fn field_satisfies(field: &Field, conds: &[FilterCond]) -> bool {
conds.iter().all(|cond| -> bool {
match cond {
FilterCond::BoolEq { path, value } => {
match path.join(".").as_str() {
"displayed.enabled" => field.displayed.enabled == *value,
"searchable.enabled" => field.searchable.enabled == *value,
"distinct.enabled" => field.distinct.enabled == *value,
"filterable.enabled" => field.filterable.enabled == *value,
"filterable.filter.equality" => field.filterable.filter.equality == *value,
"filterable.filter.comparison" => field.filterable.filter.comparison == *value,
_ => false,
}
}
FilterCond::Contains { path, value } => {
match path.join(".").as_str() {
"localized.locales" => field.localized.locales.iter().any(|l| l.contains(value)),
"filterable.facet_search.sort_by" => field.filterable.facet_search.sort_by.contains(value),
"name" => field.name.contains(value),
locale if locale.starts_with("localized.locales.") => {
let locale = locale.strip_prefix("localized.locales.").unwrap();
field.localized.locales.iter().find(|l| l == locale).is_some_and(|l| l.contains(value))
}
_ => false,
}
}
}
})
}We could add a test to ensure that we don't forget to add new fields here
There was a problem hiding this comment.
What do you think in term of readability vs performance, because I don't have the impression that performance is needed in that case.
There was a problem hiding this comment.
Not sure what's best ; both ideas make sense
| .collect(); | ||
|
|
||
| if let Some(expr) = ¶ms.filter { | ||
| let conds = parse_filter(expr); | ||
| enriched_fields.retain(|f| field_satisfies(f, &conds)); | ||
| } |
There was a problem hiding this comment.
Turning the iterator to a vec is a missed opportunity. Use filter on the iterator instead of retain on a vec, you will benefit from skip and limit from the pagination. The filter will be evaluated lazily and not all fields will be processed. Since filter isn't always set, you may need to use itertools' Either
| @@ -125,7 +129,7 @@ pub struct ListIndexes { | |||
| } | |||
|
|
|||
| impl ListIndexes { | |||
| fn as_pagination(self) -> Pagination { | |||
| fn into_pagination(self) -> Pagination { | |||
There was a problem hiding this comment.
There is no need to take ownership. According to my interpretation of Rust's naming conventions, it should be to_pagination(&self)
| @@ -580,3 +584,5 @@ pub async fn get_index_stats( | |||
| debug!(returns = ?stats, "Get index stats"); | |||
| Ok(HttpResponse::Ok().json(stats)) | |||
| } | |||
|
|
|||
| // Field-related structs, helpers and the `get_index_fields` handler have been moved to `fields.rs`. | |||
There was a problem hiding this comment.
Not sure what this is about
There was a problem hiding this comment.
Every test in this file should use snapshot instead of asserts. It will greatly simplify them
There was a problem hiding this comment.
Every test in this file should use snapshot instead of asserts. It will greatly simplify them
Add rich
/fieldsendpoint for index metadata🚀 What’s new?
New endpoint
GET /indexes/{indexUid}/fieldsReturns every field (including deep-nested ones) together with its full configuration:
Filtering & paging
offsetusize0limitusize20searchstringuser.*,*price,tags)filterstring&&Filter grammar (v1):
Examples
•
filter=displayed.enabled = true•
filter=localized.locales : fra && filterable.enabled = trueField helper flags
The response now exposes a top-level
filterable.enabledflag and an overallenabledflag at/filterable, making it trivial to testfilterable.enabled=truein the query above.Code organization
crates/meilisearch/src/routes/indexes/fields.rs.indexes::mod.rsjust re-exports the module and wires the route.Exhaustive test-suite
tests/index/fields.rs– basic behaviour (empty index, 404, search filter)tests/index/fields_recipe.rs– uploads a complex “Spaghetti Carbonara” recipe + full settings and validates:title,cuisine.type,nutrition.caloriesindex::statstests have been adapted to the new paginated format.⚙️ How to use
Basic request
Response:
Pagination
Name wildcard search
Advanced filtering
Only displayed and French-localized fields:
Combining with search
✅ Compatibility
The new module is additive; existing endpoints are untouched.
Stats tests that expected a raw array have been updated to the new JSON wrapper (
results/total/limit/offset).🔬 Implementation notes
filterparsing is deliberately simple (no OR / parentheses yet); easy to extend if needed.Feel free to ping if you need more examples or want to support additional filter operators.