Rancher compose is an undocumented legacy feature. This was originally added as a feature for RancherOS (v1.6) and is has not been maintained since. There is still code in rancher/rancher referencing this feature. This code should be removed from the rancher/rancher repo.