Skip to content

Expand tabs when parsing for USE, INCLUDE, MODULE statements #510

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 11 commits into from
Closed

Expand tabs when parsing for USE, INCLUDE, MODULE statements #510

wants to merge 11 commits into from

Conversation

urbanjost
Copy link
Contributor

Simple PR to close #508

@urbanjost urbanjost mentioned this pull request Jul 9, 2021
Closed
@LKedward LKedward self-requested a review July 9, 2021 14:25
LKedward
LKedward reviewed Jul 10, 2021
View reviewed changes
Copy link
Member

@LKedward LKedward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together quickly @urbanjost! The implementation all looks good to me, but I have a question about performance: do you have any rough idea whether this adds much overhead to reading the source files?

@urbanjost
Copy link
Contributor Author

urbanjost commented Jul 10, 2021 •
edited
Loading

The short answer is that using the general-purpose-fortran package as a test, with 1136 source files and around 282 000, lines of source it is noise level.

During a full rebuild the time is so dominated by the calls to system commands (particularly the compiler) the time in the program is insignificant, so the timing only becomes significant during a build or run|test of something essentially already built. In those cases it adds less than 1/10 of a second.

There is room, as the OP (Original Poster) noted, to combine the adjustl/lower/notabs ops at the time of reading the file, possibly removing full comment lines or lines not containing any of the keywords; with the hash having to be done first or seperately. The fpm time itself is most noticeable at a human level when doing repeated "fpm run" commands; but the system commands searching for the files still dominate that time.

Optimizing would be fruitful but also having a switch on run that bypasses the rebuild would probably be more effective at a macro level; like earlier versions did before the auto-build. But assuming you commonly want the rebuild and would install the apps the impact of speeding up fpm would be most felt when running many tests or executables at once. I did not check but I assume the rebuild check only occurs once. If it happens for each executable invocation that would add up. That just occurred to me, and easy to check.

I had not profiled everything for a while as performance has been quite satisfactory for my usage patterns; I will take a better look. I have quite a bit of profiling runs built in to some tests, and profiling numbers always have to be read aware of how cache affects I/O, and so on. But at least on Linux it added very little time.

So no one should read too much into the following gprof output of a build when everything is up to date except that the average real time barely changed (and was actually slightly faster with the expand version, which is just noise) and that the time spent in notabs for a good sized package is 0.06 seconds f ...

Rebuild on Linux with gfortran
expand  avg:: real      0m2.541s user   0m2.378s sys    0m0.172s
default avg:: real      0m2.613s user   0m2.446s sys    0m0.185s
Flat profile: expand
Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total
 time   seconds   seconds    calls  ms/call  ms/call  name
 60.87      0.42     0.42  3291871     0.00     0.00  __fpm_strings_MOD_lower
 13.04      0.51     0.09     1137     0.08     0.54  __fpm_source_parsing_MOD_parse_f_source
  8.70      0.57     0.06   283268     0.00     0.00  __fpm_strings_MOD_notabs
  4.35      0.60     0.03   306587     0.00     0.00  __fpm_strings_MOD_fnv_1a_char
  4.35      0.63     0.03     2263     0.01     0.01  __fpm_targets_MOD_find_module_dependency
  4.35      0.66     0.03     2259     0.01     0.01  __fpm_targets_MOD_add_target
  1.45      0.67     0.01    10167     0.00     0.00  __fpm_strings_MOD_split
  1.45      0.68     0.01     4518     0.00     0.00  __fpm_targets_MOD_add_dependency
  1.45      0.69     0.01     1137     0.01     0.06  __fpm_filesystem_MOD_read_lines_expanded
  0.00      0.69     0.00    23443     0.00     0.00  __fpm_filesystem_MOD_dirname
  0.00      0.69     0.00     8433     0.00     0.00  __fpm_source_parsing_MOD_split_n
Flat profile: default
Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total
 time   seconds   seconds    calls  ms/call  ms/call  name
 67.11      0.51     0.51  3291871     0.00     0.00  __fpm_strings_MOD_lower
  6.58      0.56     0.05   306587     0.00     0.00  __fpm_strings_MOD_fnv_1a_char
  6.58      0.61     0.05     2263     0.02     0.02  __fpm_targets_MOD_find_module_dependency
  6.58      0.66     0.05     1137     0.04     0.58  __fpm_source_parsing_MOD_parse_f_source
  3.95      0.69     0.03     2259     0.01     0.01  __fpm_targets_MOD_add_target
  2.63      0.71     0.02     1225     0.02     0.02  __fpm_filesystem_MOD_read_lines
  1.32      0.72     0.01     6335     0.00     0.01  __fpm_source_parsing_MOD_parse_c_source
  1.32      0.73     0.01     4649     0.00     0.00  __fpm_filesystem_MOD_canon_path
  1.32      0.74     0.01     1225     0.01     0.01  __fpm_filesystem_MOD_number_of_rows
  1.32      0.75     0.01     1210     0.01     0.05  __fpm_strings_MOD_fnv_1a_string_t
  1.32      0.76     0.01     1050     0.01     0.01  __fpm_filesystem_MOD_basename
  0.00      0.76     0.00    23443     0.00     0.00  __fpm_filesystem_MOD_dirname
  0.00      0.76     0.00    10167     0.00     0.00  __fpm_strings_MOD_split
  0.00      0.76     0.00     8433     0.00     0.00  __fpm_source_parsing_MOD_split_n

@LKedward
Copy link
Member

Thanks for the detailed reply John @urbanjost - that's good to hear. This looks good to me - I'll approve once the conflicts are resolved.

@LKedward
Copy link
Member

Thanks @urbanjost - just noticing this PR looks very different all of a sudden, did you accidently merge in a different branch? I can see lot's of changes adding fpm_stop. Are you able to open a separate PR for those?

@urbanjost urbanjost closed this Jul 18, 2021
@urbanjost urbanjost mentioned this pull request Jul 28, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@LKedward LKedward LKedward left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

FPM fails to find dependency if use statement is proceeded by tab
2 participants
@urbanjost @LKedward