Skip to content

Use implicit dotenv load #30121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 30, 2024
Merged

Conversation

mjankowski
Copy link
Contributor

This is a follow up on #30028 which reverted 18737aa

The original (reverted) commit was solving an issue where the dotenv load process was running twice -- once from the gem's own load process, and once via the explicit load call in application.rb in the app.

The issue was that the load ORDER accidentally changed with that commit. Before that, the explicit load call made sure that all the env vars were in place before the redis config file (required just below that line in application.rb) was called. After that, the file was required and set up a bunch of constants based on missing env values, since the dotenv files were not loaded yet. The dotenv files were then loaded during the initialization process so that the other (not redis config) things that rely on them were fine, and thus no specs broke, etc.

This PR repeats the removal of the dotenv load line, and moves the redis config require to a before_configuration block. I believe this creates a scenario where a) we get rid of the double load, relying just on the one from the dotenv gem, b) we make the env vars as loaded by dotenv available at the point the redis config is being loaded.

This is sort of tricky to write spec for because a) the usage/breakage is only in the development and production environments, b) the specs can only possibly run later/after the initialization phase.

I did some local experimentation with putting redis config related values into .env files and confirming via console that the relevant config was assigned -- but it would be useful to get a better specified version of "when we ran this on m.s here's what broke, and here's how to verify that in a console or server session in dev/prod environment" or something, to be sure.

@mjankowski
Copy link
Contributor Author

Separately, this redis_config could definitely be split up into multiple files since there are a few separate jobs being done by the file. The global namespace method/constants are sort of odd and could be packaged into a class or added to existing class. The sidekiq config stuff could probably just be inlined in the existing sidekiq config. The test_env_number config could probably be its own initializer, or in the test env file, etc...

@renchap
Copy link
Member

renchap commented Apr 29, 2024

it would be useful to get a better specified version of "when we ran this on m.s here's what broke

It broke on my own instance. I got those errors in the web/sidekiq processes when the cache was accessed:

RedisCacheStore: write_entry failed, returned false: Redis::CommandError: NOAUTH Authentication required.

I am using a Redis password, I assume this would not reproduce if you use the default host/port and no password.

@mjankowski
Copy link
Contributor Author

I am using a Redis password, I assume this would not reproduce if you use the default host/port and no password.

Cool ... so something like "when there is a REDIS_PASSWORD value set via a .env file, that value is preserved and run in the app's redis config?" is more or less the condition to verify here?

@renchap
Copy link
Member

renchap commented Apr 29, 2024

Yes, I think that would have caught the issue

Moves redis config require to `before_configuration` block
@mjankowski
Copy link
Contributor Author

Little mini-test along those lines. I added a REDIS_PASSWORD=test line to development env .env files...

Running in current main, but commenting out the dotenv load line, I see a) only one load (from gem, as expected), and b) output like this in console:

irb(main):002> Rails.configuration.cache_store
=> 
[:redis_cache_store,
 {:driver=>:hiredis, :url=>"redis://localhost:6379/0", :expires_in=>10 minutes, :namespace=>"cache:7.1", :connect_timeout=>5, :pool=>{:size=>5, :timeout=>5.0}}]

Note how that test value has NOT made its way into the config string.

Running on this branch, I see the same single load output from the gem doing the load, and output like:

irb(main):001> Rails.configuration.cache_store
=> 
[:redis_cache_store,
 {:driver=>:hiredis, :url=>"redis://:test@localhost:6379/0", :expires_in=>10 minutes, :namespace=>"cache:7.1", :connect_timeout=>5, :pool=>{:size=>5, :timeout=>5.0}}]

Note how test is now present.

So ... I think for the narrow goal of fixing the double load call while not breaking the redis config load, this probably works. I could see an argument for taking as-is, or for doing more cleanup work around the redis config loading first to make this issue not relevant.

Copy link
Member

@renchap renchap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, @ClearlyClaire does it makes sense to you to do it this way? I think this is what we discussed

@ClearlyClaire ClearlyClaire added this pull request to the merge queue Apr 30, 2024
Merged via the queue into mastodon:main with commit 75470f1 Apr 30, 2024
@mjankowski mjankowski deleted the redis-config-vars branch April 30, 2024 13:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants