Skip to content

[READY] Add extra conf support to Clangd completer #1267

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 26, 2019

Conversation

puremourning
Copy link
Member

@puremourning puremourning commented Jun 15, 2019

Rebased and fixed up @micbou's PR #1250

I have been using this for some time and it works well.


This change is Reviewable

@puremourning puremourning changed the title Add extra conf support to Clangd completer [READY] Add extra conf support to Clangd completer Jun 15, 2019
Copy link
Collaborator

@bstaletic bstaletic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! This will be very useful.

:lgtm:

Reviewed 15 of 15 files at r1.
Reviewable status: 1 of 2 LGTMs obtained (waiting on @puremourning)


README.md, line 105 at r1 (raw file):

for Go (using [Godef][godef] for jumping to definitions), a TSServer-based
completer for JavaScript and TypeScript, a [jdt.ls][jdtls]-based server for
Java, and a [RLS][]-based completer for Rust.  More will be added with time.

Two spaces between "Rust." and "More".

Copy link
Contributor

@kadircet kadircet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 1 of 2 LGTMs obtained (waiting on @puremourning)


ycmd/completers/cpp/clangd_completer.py, line 408 at r1 (raw file):

  def _SendFlagsFromExtraConf( self, request_data ):

I believe it make sense to perform _AddMacIncludePaths magic in here as well. WDYT?


ycmd/completers/cpp/clangd_completer.py, line 415 at r1 (raw file):

    with self._server_info_mutex:
      module = extra_conf_store.ModuleForSourceFile( filepath )

This still results in overwrite of local compilation database. Which is unnatural(at least to me) and definitely not consistent with libclang-based compiler. In libclang-based completer lookup order is as follows:

  • Use local .ycm_extra_conf.py if exists
  • Use local compile_commands.json if exists
  • Use global ycm extra conf if exists.

Current approach implementation results in:

  • Use local .ycm_extra_conf.py if exists and returns flags
  • Use global ycm extra conf if exists and returns flags.
  • Use local compile_commands.json if exists

which would both break existing users with local compilation databases and require them to write exceptions in their global ycm extra conf for each project path with a compile_commands.json in them, which is totally useless and against the principal of global ycm extra conf being a fallback.

I believe we should restore the lookup order to be same as libclang-based one.

Also I believe it is better to mimic the logic in libclang-based completer by making use of ycmd.Flags, just by making sure _GetFlagsFromExtraConfOrDatabase returns EMPTY_FLAGS in case of clangd-based completer and loads the compilation database for libclang-based completer. WDYT?

Copy link
Collaborator

@bstaletic bstaletic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 1 of 2 LGTMs obtained (waiting on @kadircet, @micbou, and @puremourning)


ycmd/completers/cpp/clangd_completer.py, line 408 at r1 (raw file):

Previously, kadircet (kadir çetinkaya) wrote…

I believe it make sense to perform _AddMacIncludePaths magic in here as well. WDYT?

If there's no other way to get clangd to discover macOS system include paths when using extra confs and no database, I don't see what choice we have.


ycmd/completers/cpp/clangd_completer.py, line 415 at r1 (raw file):

Previously, kadircet (kadir çetinkaya) wrote…

This still results in overwrite of local compilation database. Which is unnatural(at least to me) and definitely not consistent with libclang-based compiler. In libclang-based completer lookup order is as follows:

  • Use local .ycm_extra_conf.py if exists
  • Use local compile_commands.json if exists
  • Use global ycm extra conf if exists.

Current approach implementation results in:

  • Use local .ycm_extra_conf.py if exists and returns flags
  • Use global ycm extra conf if exists and returns flags.
  • Use local compile_commands.json if exists

which would both break existing users with local compilation databases and require them to write exceptions in their global ycm extra conf for each project path with a compile_commands.json in them, which is totally useless and against the principal of global ycm extra conf being a fallback.

I believe we should restore the lookup order to be same as libclang-based one.

Also I believe it is better to mimic the logic in libclang-based completer by making use of ycmd.Flags, just by making sure _GetFlagsFromExtraConfOrDatabase returns EMPTY_FLAGS in case of clangd-based completer and loads the compilation database for libclang-based completer. WDYT?

History time:

  • This proposed order used to be what we did in libclang.
  • People complained.
  • We made the breaking change and switched to the current libclang order.
  • Other people complained.

I know @micbou was against keeping the libclang order, but I think we shouldn't make that change once again.

@bstaletic
Copy link
Collaborator

One more thing. Since we never advertised that compile_flags.txt works in ycmd and never had plans to support them, let's change the clangd testdata to use extra confs instead of compile_flags.txt.

Copy link
Member Author

@puremourning puremourning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 1 of 2 LGTMs obtained (waiting on @kadircet, @micbou, and @puremourning)


ycmd/completers/cpp/clangd_completer.py, line 408 at r1 (raw file):

Previously, bstaletic (Boris Staletic) wrote…

If there's no other way to get clangd to discover macOS system include paths when using extra confs and no database, I don't see what choice we have.

Sounds like a good plan.


ycmd/completers/cpp/clangd_completer.py, line 415 at r1 (raw file):

Previously, bstaletic (Boris Staletic) wrote…

History time:

  • This proposed order used to be what we did in libclang.
  • People complained.
  • We made the breaking change and switched to the current libclang order.
  • Other people complained.

I know @micbou was against keeping the libclang order, but I think we shouldn't make that change once again.

perhaps we have to agree to disagree and allow the extra conf settings to return an indicatory to say "use compilation db if present"

Copy link
Collaborator

@bstaletic bstaletic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 1 of 2 LGTMs obtained (waiting on @kadircet, @micbou, and @puremourning)


ycmd/completers/cpp/clangd_completer.py, line 415 at r1 (raw file):

Previously, puremourning (Ben Jackson) wrote…

perhaps we have to agree to disagree and allow the extra conf settings to return an indicatory to say "use compilation db if present"

I hope you realize that, even with that extra option, ycmd will have an influx of angry users because we changed the behaviour once again and I don't want to deal with them. I also don't think another field in the dictionary won't prevent that influx.

As a user, I do prefer the current libclang order, because otherwise, when I just want the database, I need to reach into my settings and disable the global extra conf.

Note that people who want the global extra conf over a local database can do that with the current libclang ordering (place the global extra conf in $HOME), but if we revert to the old order there's no way to make the local database take priority over the global extra conf, short of completely disabling the global extra conf.

That said, I'll leave the decision to you.

Copy link
Member Author

@puremourning puremourning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 1 of 2 LGTMs obtained (waiting on @bstaletic, @kadircet, and @micbou)


README.md, line 105 at r1 (raw file):

Previously, bstaletic (Boris Staletic) wrote…

Two spaces between "Rust." and "More".

Done.


ycmd/completers/cpp/clangd_completer.py, line 415 at r1 (raw file):

Previously, bstaletic (Boris Staletic) wrote…

I hope you realize that, even with that extra option, ycmd will have an influx of angry users because we changed the behaviour once again and I don't want to deal with them. I also don't think another field in the dictionary won't prevent that influx.

As a user, I do prefer the current libclang order, because otherwise, when I just want the database, I need to reach into my settings and disable the global extra conf.

Note that people who want the global extra conf over a local database can do that with the current libclang ordering (place the global extra conf in $HOME), but if we revert to the old order there's no way to make the local database take priority over the global extra conf, short of completely disabling the global extra conf.

That said, I'll leave the decision to you.

I wasn't proposing to change the current behaviour again (that would be the ravings of a madman!), more that we make the 2 behaviours available and allow users to choose, with the default being what we currently do. Shrug.

It turns out that "just using the Flags class" is harder than it looks, but perhaps that's ripe for a refactor anyway.

Copy link
Collaborator

@bstaletic bstaletic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 1 of 2 LGTMs obtained (waiting on @kadircet and @micbou)


ycmd/completers/cpp/clangd_completer.py, line 415 at r1 (raw file):
Alright, maybe the "use comp db if present" optional field is a good idea.

It turns out that "just using the Flags class" is harder than it looks, but perhaps that's ripe for a refactor anyway.

You don't need the whole Flags class. Just the _AddMacIncludePaths( flags ) free function.

@puremourning puremourning force-pushed the clangd-completer-extra-conf branch from 99dfff1 to 46d6173 Compare June 21, 2019 17:11
@codecov
Copy link

codecov bot commented Jun 22, 2019

Codecov Report

Merging #1267 into master will increase coverage by 0.04%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #1267      +/-   ##
==========================================
+ Coverage   97.15%   97.19%   +0.04%     
==========================================
  Files          96       96              
  Lines        7376     7422      +46     
  Branches      153      153              
==========================================
+ Hits         7166     7214      +48     
+ Misses        169      167       -2     
  Partials       41       41

@codecov
Copy link

codecov bot commented Jun 22, 2019

Codecov Report

Merging #1267 into master will decrease coverage by 0.51%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #1267      +/-   ##
==========================================
- Coverage   96.95%   96.43%   -0.52%     
==========================================
  Files          96       96              
  Lines        7383     7439      +56     
  Branches      153      153              
==========================================
+ Hits         7158     7174      +16     
- Misses        184      224      +40     
  Partials       41       41

Copy link
Member Author

@puremourning puremourning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 LGTMs obtained (and 1 stale) (waiting on @bstaletic, @kadircet, and @micbou)


ycmd/completers/cpp/clangd_completer.py, line 415 at r1 (raw file):

Previously, bstaletic (Boris Staletic) wrote…

Alright, maybe the "use comp db if present" optional field is a good idea.

It turns out that "just using the Flags class" is harder than it looks, but perhaps that's ripe for a refactor anyway.

You don't need the whole Flags class. Just the _AddMacIncludePaths( flags ) free function.

  • used the same logic for extra conf/database/global as Flags
  • Added Mac flags and a test. Also I changed the debug info to include the actual thing we send to clangd, which is much more valuable.

Copy link
Member Author

@puremourning puremourning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 LGTMs obtained (and 1 stale) (waiting on @bstaletic, @kadircet, and @micbou)


ycmd/completers/cpp/clangd_completer.py, line 415 at r1 (raw file):

Previously, puremourning (Ben Jackson) wrote…
  • used the same logic for extra conf/database/global as Flags
  • Added Mac flags and a test. Also I changed the debug info to include the actual thing we send to clangd, which is much more valuable.

Done.

@puremourning
Copy link
Member Author

I believe we should restore the lookup order to be same as libclang-based one.

@kadircet - done as above.

I tested the Mac include thing and it makes clangd work on my computers! \o/

@puremourning puremourning force-pushed the clangd-completer-extra-conf branch from f3df502 to 46884e5 Compare June 22, 2019 16:01
Copy link
Collaborator

@bstaletic bstaletic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, but the tests are failing on Windows.

Reviewed 3 of 3 files at r2, 1 of 2 files at r3, 4 of 4 files at r4.
Reviewable status: 0 of 2 LGTMs obtained (and 1 stale) (waiting on @kadircet, @micbou, and @puremourning)


ycmd/completers/cpp/clangd_completer.py, line 442 at r4 (raw file):

        return

      if ( settings.get( 'do_cache', True ) and

Is there any reason for a user to have do_cache set to False? I don't see any and it is simpler to just assume True.

Copy link
Member Author

@puremourning puremourning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 LGTMs obtained (and 1 stale) (waiting on @bstaletic, @kadircet, and @micbou)


ycmd/completers/cpp/clangd_completer.py, line 442 at r4 (raw file):

Previously, bstaletic (Boris Staletic) wrote…

Is there any reason for a user to have do_cache set to False? I don't see any and it is simpler to just assume True.

We do assume true, that's what the second argument to get does (returns True if it's not set or not present)

Copy link
Collaborator

@bstaletic bstaletic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 LGTMs obtained (and 1 stale) (waiting on @kadircet, @micbou, and @puremourning)


ycmd/completers/cpp/clangd_completer.py, line 442 at r4 (raw file):

Previously, puremourning (Ben Jackson) wrote…

We do assume true, that's what the second argument to get does (returns True if it's not set or not present)

That's a different definition of "assume". I was thinking of just skipping the settings.get( 'do_cache', True ) part and ignore whatever the user sets for do_cache.

Strictly speaking, it is a breaking change if a user really wants do_cache to be false, but... why would anyone want that?

Copy link
Member Author

@puremourning puremourning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 LGTMs obtained (and 1 stale) (waiting on @kadircet and @micbou)


ycmd/completers/cpp/clangd_completer.py, line 442 at r4 (raw file):

Previously, bstaletic (Boris Staletic) wrote…

That's a different definition of "assume". I was thinking of just skipping the settings.get( 'do_cache', True ) part and ignore whatever the user sets for do_cache.

Strictly speaking, it is a breaking change if a user really wants do_cache to be false, but... why would anyone want that?

I suspect @micbou was trying to obey the ycmd docs for this (unusual) flag. It's not a big deal for me. We don't have a test for it, but we don't have a test for it with the clang completer (probably!) so I say it's harmless to leave it in.

@puremourning puremourning force-pushed the clangd-completer-extra-conf branch from 46884e5 to b907d8f Compare June 23, 2019 20:21
Copy link
Contributor

@kadircet kadircet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Reviewable status: 0 of 2 LGTMs obtained (and 1 stale) (waiting on @bstaletic, @micbou, and @puremourning)


ycmd/completers/cpp/clangd_completer.py, line 425 at r6 (raw file):

      module = extra_conf_store.ModuleForSourceFile( filepath )
      if module and not extra_conf_store.IsGlobalExtraConfModule( module ):

nit: maybe simplify a bit with:

# no extra conf, let clangd fallback
if not module:
  return

# global ycm extra conf and local comp db, let clangd use local comp db
if extra_conf_store.IsGlobalExtraConfModule( module ) and CompilationDatabaseExists( filepath ):
  return

# use ycm_extra_conf either local or global
settings = self.GetSettings( module, request_data )

Copy link
Contributor

@kadircet kadircet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: 1 of 2 LGTMs obtained (and 1 stale) (waiting on @bstaletic, @micbou, and @puremourning)

Copy link
Member Author

@puremourning puremourning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 LGTMs obtained (and 2 stale) (waiting on @bstaletic and @kadircet)


ycmd/completers/cpp/clangd_completer.py, line 425 at r6 (raw file):

Previously, kadircet (kadir çetinkaya) wrote…

nit: maybe simplify a bit with:

# no extra conf, let clangd fallback
if not module:
  return

# global ycm extra conf and local comp db, let clangd use local comp db
if extra_conf_store.IsGlobalExtraConfModule( module ) and CompilationDatabaseExists( filepath ):
  return

# use ycm_extra_conf either local or global
settings = self.GetSettings( module, request_data )

Good point. I was trying to mirror the flags.py logic too closely.

Done.

@puremourning puremourning force-pushed the clangd-completer-extra-conf branch from b7e5d91 to f2ad374 Compare June 26, 2019 07:10
@codecov-io
Copy link

codecov-io commented Jun 26, 2019

Codecov Report

Merging #1267 into master will decrease coverage by 0.51%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #1267      +/-   ##
==========================================
- Coverage   96.95%   96.43%   -0.52%     
==========================================
  Files          96       96              
  Lines        7383     7439      +56     
  Branches      153      153              
==========================================
+ Hits         7158     7174      +16     
- Misses        184      224      +40     
  Partials       41       41

Copy link
Collaborator

@bstaletic bstaletic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm_strong:, now that the tests work.
Consider squashing the commits. Or just merge the PR.

Codecov seems to be confused, hopefully its data isn't corrupted and it's just another hiccup.

Reviewed 1 of 6 files at r5, 5 of 5 files at r6, 1 of 1 files at r7, 5 of 5 files at r9.
Reviewable status: 1 of 2 LGTMs obtained (and 1 stale) (waiting on @kadircet)

puremourning and others added 2 commits June 26, 2019 14:04
Use the same logic as libclang: local conf, database, then global. Also
add Mac Include paths, but don't do most of the other mitigations that
we do for direct libclang.
@puremourning puremourning force-pushed the clangd-completer-extra-conf branch from e0e3afb to eed9ca6 Compare June 26, 2019 13:05
Copy link
Member Author

@puremourning puremourning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rebased and squashed.

Reviewed 1 of 5 files at r9, 16 of 16 files at r11.
Reviewable status: 1 of 2 LGTMs obtained (and 1 stale)

@puremourning
Copy link
Member Author

Greeeeeeen!

@puremourning puremourning added the Ship It! Manual override to merge a PR by maintainer label Jun 26, 2019
@mergify mergify bot merged commit f16d443 into ycm-core:master Jun 26, 2019
@mergify
Copy link
Contributor

mergify bot commented Jun 26, 2019

Thanks for sending a PR!

@puremourning puremourning deleted the clangd-completer-extra-conf branch June 26, 2019 15:40
bstaletic added a commit to bstaletic/YouCompleteMe that referenced this pull request Jul 3, 2019
ycm-core/ycmd#1275 Remap python GoTo* commands
ycm-core/ycmd#1274 Migrate to Omnisharp-Roslyn
ycm-core/ycmd#1239 Error ID in diagnostic messages
ycm-core/ycmd#1267 Clangd extra conf support
ycm-core/ycmd#1266 JDT update to 0.40.0
ycm-core/ycmd#1245 Generic (pluggable) LSP completer support
ycm-core/ycmd#1263 Support JDT extensions
ycm-core/ycmd#1261 Typescript update
ycm-core/ycmd#1264 [vimspector](/puremourning/vimspector) configuration
ycm-core/ycmd#1262 Multi-user installation
ycm-core/ycmd#1243 Migrate to GOPLS
ycm-core/ycmd#1249 Support MSVC 16 (VS2019)
ycm-core/ycmd#1224 Migrate to RLS
bstaletic added a commit to bstaletic/YouCompleteMe that referenced this pull request Jul 4, 2019
ycm-core/ycmd#1275 Remap python GoTo* commands
ycm-core/ycmd#1274 Migrate to Omnisharp-Roslyn
ycm-core/ycmd#1239 Error ID in diagnostic messages
ycm-core/ycmd#1267 Clangd extra conf support
ycm-core/ycmd#1266 JDT update to 0.40.0
ycm-core/ycmd#1245 Generic (pluggable) LSP completer support
ycm-core/ycmd#1263 Support JDT extensions
ycm-core/ycmd#1261 Typescript update
ycm-core/ycmd#1264 [vimspector](/puremourning/vimspector) configuration
ycm-core/ycmd#1262 Multi-user installation
ycm-core/ycmd#1243 Migrate to GOPLS
ycm-core/ycmd#1249 Support MSVC 16 (VS2019)
ycm-core/ycmd#1224 Migrate to RLS
bstaletic added a commit to bstaletic/YouCompleteMe that referenced this pull request Jul 4, 2019
ycm-core/ycmd#1275 Remap python GoTo* commands
ycm-core/ycmd#1274 Migrate to Omnisharp-Roslyn
ycm-core/ycmd#1239 Error ID in diagnostic messages
ycm-core/ycmd#1267 Clangd extra conf support
ycm-core/ycmd#1266 JDT update to 0.40.0
ycm-core/ycmd#1245 Generic (pluggable) LSP completer support
ycm-core/ycmd#1263 Support JDT extensions
ycm-core/ycmd#1261 Typescript update
ycm-core/ycmd#1264 [vimspector](/puremourning/vimspector) configuration
ycm-core/ycmd#1262 Multi-user installation
ycm-core/ycmd#1243 Migrate to GOPLS
ycm-core/ycmd#1249 Support MSVC 16 (VS2019)
ycm-core/ycmd#1224 Migrate to RLS
bstaletic added a commit to bstaletic/YouCompleteMe that referenced this pull request Jul 4, 2019
ycm-core/ycmd#1275 Remap python GoTo* commands
ycm-core/ycmd#1274 Migrate to Omnisharp-Roslyn
ycm-core/ycmd#1239 Error ID in diagnostic messages
ycm-core/ycmd#1267 Clangd extra conf support
ycm-core/ycmd#1266 JDT update to 0.40.0
ycm-core/ycmd#1245 Generic (pluggable) LSP completer support
ycm-core/ycmd#1263 Support JDT extensions
ycm-core/ycmd#1261 Typescript update
ycm-core/ycmd#1264 [vimspector](/puremourning/vimspector) configuration
ycm-core/ycmd#1262 Multi-user installation
ycm-core/ycmd#1243 Migrate to GOPLS
ycm-core/ycmd#1249 Support MSVC 16 (VS2019)
ycm-core/ycmd#1224 Migrate to RLS
bstaletic added a commit to bstaletic/YouCompleteMe that referenced this pull request Jul 5, 2019
ycm-core/ycmd#1275 Remap python GoTo* commands
ycm-core/ycmd#1274 Migrate to Omnisharp-Roslyn
ycm-core/ycmd#1239 Error ID in diagnostic messages
ycm-core/ycmd#1267 Clangd extra conf support
ycm-core/ycmd#1266 JDT update to 0.40.0
ycm-core/ycmd#1245 Generic (pluggable) LSP completer support
ycm-core/ycmd#1263 Support JDT extensions
ycm-core/ycmd#1261 Typescript update
ycm-core/ycmd#1264 [vimspector](/puremourning/vimspector) configuration
ycm-core/ycmd#1262 Multi-user installation
ycm-core/ycmd#1243 Migrate to GOPLS
ycm-core/ycmd#1249 Support MSVC 16 (VS2019)
ycm-core/ycmd#1224 Migrate to RLS
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Ship It! Manual override to merge a PR by maintainer
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants