There are wp_enqueue_block_style function that allow load custom styles per block from CSS files. It's relatively easy to implement such loading using PHP but many WP users are not familiar with programming. So, there are idea how to do it via theme.json. There are "css" propery in theme.json that can be defined inside every block section. Now it can accept only raw css body. I suggest to extend this property to accept also paths to CSS files like this:

"core/navigation": {				
	"css": "file:./assets/stylesheets/blocks/navigation.css"
	"color": {
		"background": "var:preset|color|contrast-3"
	},
	"spacing": {
		"blockGap": "0"
	},
	"typography": {
		"fontWeight": "var:custom|font-weight|semi-bold",
		"textTransform": "uppercase"
	}
}

This allows to enqueue CSS style for core/navigation block from ../assets/stylesheets/blocks/navigation.css file automatically using wp_enqueue_block_style under the hood.