Skip to content

Conversation

siriwatknp
Copy link
Member

@siriwatknp siriwatknp commented Jun 30, 2025

Preview

This PR updates the LLM documentation generation to create better organized and more comprehensive documentation files:

Root Index:

Project Indexes:

Summary

This PR enhances the LLM documentation generator to properly include API documentation in project indexes and improves the overall organization of generated documentation files.

The generated llms.txt has the same structure as the docs sidebar.

How to run/test

To test the changes and generate the updated documentation:

# Generate LLM documentation for all projects
pnpm tsx scripts/buildLlmsDocs/index.ts --projectSettings ./scripts/buildApiDocs/projectSettings.ts

# Generate for specific projects only
pnpm tsx scripts/buildLlmsDocs/index.ts --projectSettings ./scripts/buildApiDocs/projectSettings.ts --grep "charts|pickers"

# Verify the output files
ls -la docs/public/x/
ls -la docs/public/x/react-charts/
ls -la docs/public/x/react-date-pickers/

Expected results:

  • Root docs/public/x/llms.txt should show clean titles without project prefixes
  • Each project's llms.txt should include API documentation in the Resources section
  • API files should be properly organized under docs/public/x/api/{project}/

Review Notes

🔍 Key Areas to Review

  1. API Integration Logic (lines 922-937): Check how the regex pattern finds API sections and processes them
  2. File Processing Simplification (lines 1009-1050): Verify the removal of complex grouping logic works correctly
  3. Title Prefix Removal (lines 393-395): Ensure the regex correctly strips project prefixes without affecting other content

📁 Output Structure

docs/public/x/
├── llms.txt                    # Root index with clean titles
├── react-charts/
│   └── llms.txt               # Includes API files in Resources section
├── react-data-grid/
│   └── llms.txt               # Includes API files in Resources section
└── api/
    ├── charts/
    │   ├── bar-chart.md
    │   └── line-chart.md
    └── data-grid/
        └── grid-api.md

Breaking Changes

None - this is purely an enhancement to the documentation generation process.

siriwatknp and others added 16 commits June 27, 2025 10:06
- Modified findChildrenByTitle to accept regex patterns
- Removed unnecessary groupedByFirstDir logic
- Pass all project files directly to generateProjectLlmsTxt
- This ensures API files are properly included in the Resources section

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Remove unnecessary matchGeneratedFilesWithPages function and inline the logic
- Remove generateSimpleProjectLlmsTxt fallback function
- Throw error if pages section is not found instead of silently falling back
- This simplifies the codebase and ensures consistent behavior

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Updated function comments to reflect actual functionality
- Removed outdated placeholder comments and logging statements
- Updated file path examples to match current output structure
- Simplified verbose comments while maintaining clarity
- Changed CLI description to "MUI X components" for accuracy

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@siriwatknp siriwatknp added docs Improvements or additions to the documentation. scope: docs-infra Changes related to the docs-infra product. labels Jun 30, 2025
@mui-bot
Copy link

mui-bot commented Jun 30, 2025

Deploy preview: https://deploy-preview-18595--material-ui-x.netlify.app/

Bundle size report

Total Size Change: 0B(0.00%) - Total Gzip Change: ▼-7B(0.00%)
Files: 122 total (0 added, 0 removed, 6 changed)

Show details for 100 more bundles (22 more not shown)

@mui/x-chartsparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts-proparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts-pro/BarChartProparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts-pro/ChartContainerProparsed: 0B(0.00%) gzip: ▼-2B(0.00%)
@mui/x-charts-pro/ChartDataProviderProparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts-pro/ChartsToolbarProparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts-pro/ChartZoomSliderparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts-pro/FunnelChartparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts-pro/Heatmapparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts-pro/LineChartProparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts-pro/PieChartProparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts-pro/RadarChartProparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts-pro/ScatterChartProparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/BarChartparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/ChartContainerparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/ChartDataProviderparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/ChartsAxisparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/ChartsAxisHighlightparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/ChartsClipPathparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/ChartsGridparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/ChartsLabelparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/ChartsLegendparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/ChartsLocalizationProviderparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/ChartsOverlayparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/ChartsReferenceLineparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/ChartsSurfaceparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/ChartsTextparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/ChartsTooltipparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/ChartsXAxisparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/ChartsYAxisparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/Gaugeparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/LineChartparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/PieChartparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/RadarChartparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/ScatterChartparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/SparkLineChartparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-charts/Toolbarparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-data-gridparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-data-grid-premiumparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-data-grid-premium/DataGridPremiumparsed: 0B(0.00%) gzip: ▼-3B(0.00%)
@mui/x-data-grid-proparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-data-grid-pro/DataGridProparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-data-grid/DataGridparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickersparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-proparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/AdapterDateFnsparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/AdapterDateFnsJalaliparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/AdapterDayjsparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/AdapterLuxonparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/AdapterMomentparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/AdapterMomentHijriparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/AdapterMomentJalaaliparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/DateRangeCalendarparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/DateRangePickerparsed: 0B(0.00%) gzip: ▼-1B(0.00%)
@mui/x-date-pickers-pro/DateRangePickerDayparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/DateRangePickerDay2parsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/DateTimeRangePickerparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/DesktopDateRangePickerparsed: 0B(0.00%) gzip: 🔺+1B(0.00%)
@mui/x-date-pickers-pro/DesktopDateTimeRangePickerparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/DesktopTimeRangePickerparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/LocalizationProviderparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/MobileDateRangePickerparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/MobileDateTimeRangePickerparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/MobileTimeRangePickerparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/MultiInputDateRangeFieldparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/MultiInputDateTimeRangeFieldparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/MultiInputTimeRangeFieldparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/PickersRangeCalendarHeaderparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/SingleInputDateRangeFieldparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/SingleInputDateTimeRangeFieldparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/SingleInputTimeRangeFieldparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/StaticDateRangePickerparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers-pro/TimeRangePickerparsed: 0B(0.00%) gzip: ▼-1B(0.00%)
@mui/x-date-pickers/AdapterDateFnsparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/AdapterDateFnsBaseparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/AdapterDateFnsJalaliparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/AdapterDayjsparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/AdapterLuxonparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/AdapterMomentparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/AdapterMomentHijriparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/AdapterMomentJalaaliparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/DateCalendarparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/DateFieldparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/DatePickerparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/DateTimeFieldparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/DateTimePickerparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/DayCalendarSkeletonparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/DesktopDatePickerparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/DesktopDateTimePickerparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/DesktopTimePickerparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/DigitalClockparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/LocalizationProviderparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/MobileDatePickerparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/MobileDateTimePickerparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/MobileTimePickerparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/MonthCalendarparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/MultiSectionDigitalClockparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/PickerDay2parsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/PickersActionBarparsed: 0B(0.00%) gzip: 0B(0.00%)
@mui/x-date-pickers/PickersCalendarHeaderparsed: 0B(0.00%) gzip: 0B(0.00%)

Details of bundle changes

Generated by 🚫 dangerJS against 9f10e8f

@siriwatknp siriwatknp requested a review from bharatkashyap July 1, 2025 02:39
Copy link
Member

@alexfauquette alexfauquette left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems their is an issue with data gird nested pages. In llm.txt the server side data section only have the overview page.

image

Pages are not added because those page definition have no title, and the following line in the code set matchedFile to undefined

const matchedFile = fileMap.get(page.pathname);

@siriwatknp
Copy link
Member Author

It seems their is an issue with data gird nested pages. In llm.txt the server side data section only have the overview page.

I'll fix it, thanks for pointing that out.

siriwatknp and others added 2 commits July 1, 2025 16:14
Co-authored-by: Alexandre Fauquette <45398769+alexfauquette@users.noreply.github.com>
Signed-off-by: Siriwat K <siriwatkunaporn@gmail.com>
@alexfauquette
Copy link
Member

alexfauquette commented Jul 7, 2025

I see few potentail improvment that could be done in a follow up PR

  • How to skip some markdown pages? Mostly the overview pages that are markdown but contains no information. It's already the case for pickers
    I just merger the PR doing the same for charts

  • Some remaining {{"xxx": ....

    • Some demo do not get translated. A bunch of them are in react-charts/radar.md.
    • data grid has the SelectorsDocs components. We could open an issue to transform them in a md files later.
    • The ComponentLinkHeader could probably just be removed
  • Is it intentional to have conde snipets with both jsx and tsx format? I would assume tsx is enought

  • For demo with multiple files, only the main one is added in the md. Should we had the with on top of the snippet a comment with the file name? Somtheing like // ./CustomeTooltipContainer.tsx. This open other question because in charts we also use dedicated files for dataset. SO you probably don't what to polute the md files with the evolution Germany's GDP over the last 20 years, or Meta's stock price 🤭

Some other remark that might be harder to do:

  • The installation instruction are missing, because we use things like ChartsInstallationInstructions to allow user to select what they are looking for an get the corresponding installation command. The solution I can think about is to add comments in the installation files that should be copy pasted by the script. SOmething like
/** LLM files instruction
 * To install community package
 * ```bash
 * pnpm install @mui/x-chart
 * ```
 */

Or the other option is we don't care because installing a package is trivial for bots.

  • Do we have a plan to maintain those files? I one year how do we detect that conventions used in this script are still respected by the team?

Copy link
Member

@alexfauquette alexfauquette left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving because the script looks good. My comments are more followup potential refinements

@siriwatknp
Copy link
Member Author

siriwatknp commented Jul 8, 2025

How to skip some markdown pages? Mostly the overview pages that are markdown but contains no information

I think this can be ignore in this PR. I don't want to add more logic to the script if those files are minimal. If it impact the quality of code output, we can revisit.

Some remaining

Will be fixed by mui/material-ui#46494

Is it intentional to have conde snipets with both jsx and tsx format? I would assume tsx is enought

Agree, only tsx is enough (fallback to js if not exist). Will be fixed by mui/material-ui#46494

@siriwatknp
Copy link
Member Author

Some other remark that might be harder to do:

True, those small details are too complex and might not be worth fixing now. The script will be too complex.

Do we have a plan to maintain those files? I one year how do we detect that conventions used in this script are still respected by the team?

In my opinion, likely yes as the llms.txt becomes a standard. We will need to experiment with the quality of the code generated from these llms.txt until we make more improvements/decision to continue.

@github-actions github-actions bot added the PR: out-of-date The pull request has merge conflicts and can't be merged. label Jul 9, 2025
Copy link

github-actions bot commented Jul 9, 2025

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@github-actions github-actions bot removed the PR: out-of-date The pull request has merge conflicts and can't be merged. label Jul 9, 2025
@siriwatknp siriwatknp merged commit 7ab76f0 into mui:master Jul 9, 2025
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Improvements or additions to the documentation. scope: docs-infra Changes related to the docs-infra product.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants