-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
[code-infra] Auto-generate deep exports to prevent asymmetric exports #18917
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Localization writing tips ✍️Seems you are updating localization 🌍 files. Thank you for contributing to the localization! 🎉 To make your PR perfect, here is a list of elements to check: ✔️
Deploy preview: https://deploy-preview-18917--material-ui-x.netlify.app/ Bundle size report
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great initiative 👍
scripts/generateDeepExports.mjs
Outdated
const utilityDirs = [ | ||
'internals', | ||
'context', | ||
'hooks', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this be configured by package?
Also, shouldn't we remove hooks
from this?
For example, the SeriesDemo
uses deep imports from the hooks
directory: import { useLineSeries, useXAxis, useXScale, useYScale } from '@mui/x-charts/hooks';
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤔 I assumed we were already exporting hooks from pro/hooks/index.ts
but we are not. However the file exists, so I would rather we just add it there.
This is a naive approach, so it still requires us to do some work to ensure everything is correct
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, this is already a step forward.
I wonder if we should have a auto generated section in a file instead.
We can use something like:
// User code here
// # region auto-generated
// Generated code here
// # endregion
So we can gradually move the hooks
export to auto generated ones.
Something for the future, though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could, but it also is a good recipe to create conflicts 🫠
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll try to do it for specific folders instead. Like context/hooks
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've made it configurable and allow block comment in specific utilities packages.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
but it also is a good recipe to create conflicts 🫠
Agree.
TLTR; Instead of having this feature, we could add the missing export * from ...
manually and remove this notion of utilities
. We would end up in the same state.
I don't see why utils should be treated differently from components. For me there are two categories
- stuff users need to import, and we should be consistent
- stuff users don't care about, and we can ignore them.
Imagine you add a new utils folder.
Step 1: pro and premium have nothing to add. The autogenerated index.ts
can be the same as the one for component
Step 2: pro needs to add an extra function. We remove the comments but keep the export * from ...
. Now the file is ignored by the script and we are responsible for it. But not removing an export * from ...
shoudl be doable :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think now it is implemented we might as well have it, since I added other functionalities to the utilities as well 😢
@@ -57,6 +57,7 @@ | |||
"release:publish:dry-run": "pnpm publish --recursive --tag latest --registry=\"http://localhost:4873/\"", | |||
"release:tag": "node scripts/releaseTag.mjs", | |||
"release:prepare": "node scripts/createReleasePR.mjs", | |||
"generate:exports": "node scripts/generateDeepExports.mjs", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of checking in the CI, why not call this script from the build
command directly? This will avoid requiring users to run the script before committing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm just following our current approach of committing the generated stuff rather than having it only on build-time.
If @mui/infra wants to go that way I could happily switch 😆
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would probably not work too well if we only generate on build-time though 🤔, as the imports wouldn't be available in, lets say, the docs pages... 😢
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO the approach in this PR is better. It's easier to gain more control over it when it's part of source code, and it makes the build less magic.
scripts/generateDeepExports.mjs
Outdated
|
||
return ( | ||
fs | ||
.readdirSync(srcPath, { withFileTypes: true }) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As a general rule, I would prefer it if we stopped using blocking io everywhere.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried it out, but I think it makes sense for this instance to use sync funcs. Since the behaviour depends on changes made by this script, eg x-charts-premium
needs to know the changes done to x-charts-pro
, so we have to do one package after another.
We could have async read/write inside a given package, but the async code is much more verbose for this specific implementation 😓
This script runs super fast anyways, so it might not be needed to optimise
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can keep processing the packages sequentially, but so much file access can run parallel in this PR. It's hardly more verbose with async/await if done properly. Most of the calls here can just have their non-blocking counterpart with an await. just the few loops have to be a replaced wit a Promise.all
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice one
// This re-export-block is automatically generated, to customize, simply remove the block. | ||
export * from '@mui/x-charts/colorPalettes'; | ||
// End of re-export-block |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this a block comment and not just
// Re-export automatically generated, to customize, simply remove this line.
like for previouse ones?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I finally got it. It's because the folder is part of the utilities
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I wanted to have a full file override mode and a partial one. The idea is that some files are more often customised, like hooks
, while others are unlikely to be reimplemented, like BarChart
scripts/generateDeepExports.mjs
Outdated
const utilityDirs = [ | ||
'internals', | ||
'context', | ||
'hooks', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
but it also is a good recipe to create conflicts 🫠
Agree.
TLTR; Instead of having this feature, we could add the missing export * from ...
manually and remove this notion of utilities
. We would end up in the same state.
I don't see why utils should be treated differently from components. For me there are two categories
- stuff users need to import, and we should be consistent
- stuff users don't care about, and we can ignore them.
Imagine you add a new utils folder.
Step 1: pro and premium have nothing to add. The autogenerated index.ts
can be the same as the one for component
Step 2: pro needs to add an extra function. We remove the comments but keep the export * from ...
. Now the file is ignored by the script and we are responsible for it. But not removing an export * from ...
shoudl be doable :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure why the new GitHub UI doesn't let me just leave comments, I need to write this comment before it lets me comment 😅
scripts/generateDeepExports.mjs
Outdated
if (!existsSync(srcPath)) { | ||
return []; | ||
} | ||
|
||
const dirents = await fs.readdir(srcPath, { withFileTypes: true }); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be better to readdir
and handle the ENOENT
error if the directory doesn't exist?
It's one less call. For a lot of files it might be faster.
Same applies on line 189, and other lines across this script.
Not a blocker, though. Feel free to do it as a follow-up if you prefer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Plus it's vulnerable to race conditions. Another program may remove the dir in between the exists
check and the readdir
call. So you technically have to handle that error anyway. At which point the exists
check becomes unnecessary. I believe this would be one of the reasons why exists
was deprecated in the first place. It was mostly used in patterns that are vulnerable to race conditions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another program may remove the dir in between the
exists
check and thereaddir
call
Yeah, I didn't add that because I think it's unlikely to happen in this case, but it's true it's a possibility
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's one less call. For a lot of files it might be faster.
The script is quite fast, I don't think there is any concern in that aspect. I've refactored to use catch blocks instead on existsSync however.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nitpick, but those catch blocks could potentially hide other errors than just the missing dir.
} | ||
|
||
// Get additional components specific to pro/premium | ||
// eslint-disable-next-line no-await-in-loop |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we follow this lint? It'd probably make the script slightly faster by processing files concurrently.
Also not a blocker, as it doesn't affect the functionality of the script.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, these need to run in order
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, you mean because the premium package depends on the generated exports of the pro package?
Yeah, makes sense. We could have some concurrency here, but probably not worth the extra complexity.
scripts/x-charts-pro.exports.json
Outdated
@@ -204,7 +207,13 @@ | |||
{ "name": "DefaultizedSeriesType", "kind": "TypeAlias" }, | |||
{ "name": "defaultOnBeforeExport", "kind": "Function" }, | |||
{ "name": "Direction", "kind": "TypeAlias" }, | |||
{ "name": "elGR", "kind": "Variable" }, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't we avoid exporting the locales from the index file?
When someone is doing import { LineChartPro } from '@mui/x-charts-pro'
, I believe these locales will still need to be evaluated by the bundler, even if they're dropped in the end. @Janpot can probably confirm here.
We aren't exporting the locales from the community package.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some bundlers like vite would have some overhead in dev mode with barrel files. Next.js shouldn't be impacted as they have built special handling of these barrel files. IMO, until vite fixes that issue, we should just advice performance-conscious users to not user the barrel file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've reverted it to be in line with our current approach
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left comments in the index files as well
// Locales should be imported from `@mui/x-charts-premium/locales` | ||
// export * from './locales'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we remove this? Why keep it commented?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So we don't make the same mistake again
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure if it'll be very effective, but I'm ok with it
@@ -1,12 +1,9 @@ | |||
// TODO: Add typeOverloads when available | |||
// import './typeOverloads/modules'; | |||
// eslint-disable-next-line no-restricted-imports | |||
import '@mui/x-charts-pro/typeOverloads/modules'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should the type overloads from the premium package import the pro ones instead so that if someone imports @mui/x-charts-premium/typeOverloads
, they get the pro ones as well?
Otherwise, they'd need to import the pro and premium type overloads in sequence.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently there is no premium typeOverloads. But yes, I've left the TODO:
there to do that eventually
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
Signed-off-by: Jose C Quintas Jr <juniorquintas@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great job 💯
This PR tries to fix asymmetric exports in community and pro+ packages
Eg: If you are using a
PieChart
on community, you would use it like thisBut then on pro, you import it from the root, because the is no
src/PieChart
folder in the pro package.This change is a naive implementation that allows users to deep import, as