Skip to content

Conversation

byroot
Copy link
Member

@byroot byroot commented Dec 9, 2022

Use case

A very common pattern in Ruby, especially in testing is to save the value of an attribute, set a new value, and then restore the old value in an ensure clause.

e.g. in unit tests

def test_something_when_enabled
  enabled_was, SomeLibrary.enabled = SomeLibrary.enabled, true
  # test things
ensure
  SomeLibrary.enabled = enabled_was
end

Or sometime in actual APIs:

def with_something_enabled
  enabled_was = @enabled
  @enabled = true
  yield
ensure
  @enabled = enabled_was
end

There is no inherent problem with this pattern, but it can be easy to make a mistake, for instance the unit test example:

def test_something_when_enabled
  some_call_that_may_raise
  enabled_was, SomeLibrary.enabled = SomeLibrary.enabled, true
  # test things
ensure
  SomeLibrary.enabled = enabled_was
end

In the above if some_call_that_may_raise actually raises, SomeLibrary.enabled is set back to nil rather than its original value. I've seen this mistake quite frequently.

Object#with

I think it would be very useful to have a method on Object to implement this pattern in a correct and easy to use way.

NB: public_send is used because I don't think such method should be usable if the accessors are private.

With usage:

def test_something_when_enabled
  SomeLibrary.with(enabled: true) do
    # test things
  end
end
GC.with(measure_total_time: true, auto_compact: false) do
  # do something
end

Lots of tests in Rails's codebase could be simplified, e.g.:

  • Changing Thread.report_on_exception:
    def test_non_bang_disconnect_and_clear_reloadable_connections_throw_exception_if_threads_dont_return_their_conns
    Thread.report_on_exception, original_report_on_exception = false, Thread.report_on_exception
    @pool.checkout_timeout = 0.001 # no need to delay test suite by waiting the whole full default timeout
    [:disconnect, :clear_reloadable_connections].each do |group_action_method|
    @pool.with_connection do |connection|
    assert_raises(ExclusiveConnectionTimeoutError) do
    new_thread { @pool.public_send(group_action_method) }.join
    end
    end
    end
    ensure
    Thread.report_on_exception = original_report_on_exception
    end
  • Changing a class attribute:
    def test_optional_relation
    original_value = ActiveRecord::Base.belongs_to_required_by_default
    ActiveRecord::Base.belongs_to_required_by_default = true
    model = Class.new(ActiveRecord::Base) do
    self.table_name = "accounts"
    def self.name; "Temp"; end
    belongs_to :company, optional: true
    end
    account = model.new
    assert_predicate account, :valid?
    ensure
    ActiveRecord::Base.belongs_to_required_by_default = original_value
    end

@fatkodima
Copy link
Member

def with_something_enabled
  enabled_was = @enabled
  @enabled = true
  yield
ensure
  @enabled = enabled_was
end

Looking at the code, something like this is not currently supported?

with(:@enabled => true)
  yield
end

Found many examples of this in the rails codebase itself. For example,

def in_rendering_context(options)
old_view_renderer = @view_renderer
old_lookup_context = @lookup_context
if !lookup_context.html_fallback_for_js && options[:formats]
formats = Array(options[:formats])
if formats == [:js]
formats << :html
end
@lookup_context = lookup_context.with_prepended_formats(formats)
@view_renderer = ActionView::Renderer.new @lookup_context
end
yield @view_renderer
ensure
@view_renderer = old_view_renderer
@lookup_context = old_lookup_context
end
or
def test_redirect_to_with_adding_flash_types
original_controller = @controller
test_controller_with_flash_type_foo = Class.new(TestController) do
add_flash_types :foo
end
@controller = test_controller_with_flash_type_foo.new
get :redirect_with_foo_flash
assert_equal "for great justice", @controller.flash[:foo]
ensure
@controller = original_controller
end

@byroot
Copy link
Member Author

byroot commented Dec 9, 2022

something like this is not currently supported?

Nope, and I don't wish to, similarly to why it only allow to use public methods, instance variables are out of scope.

Otherwise we'd open a huge way to bypass encapsulation.

@jonathanhefner
Copy link
Member

What do you think about adding require "active_support/core_ext/object/with" to active_support/test_case.rb, so that #with will be automatically available wherever ActiveSupport::TestCase is?

On one hand, active_support/test_case.rb doesn't include any other such extensions. But, on the other hand, #with is arguably tangential to active_support/testing/setup_and_teardown.

@byroot
Copy link
Member Author

byroot commented Dec 10, 2022

Just to clarify, this is a very common pattern in test but not only. It isn't meant as purely a test helper, otherwise AS::TestCase#with_attr(object, **attributes) would do just fine without adding to Object.

I think this is also commonly useful in production code, e.g.

Rails.logger.with(level: Logger::Debug) do
  # ...
end

etc.

@jonathanhefner
Copy link
Member

Sure! I didn't mean to imply it was only for tests. I was just thinking of all the places it would be useful in Rails' own tests. But we could also add the require to test helper files (e.g. abstract_unit.rb) instead.

@simi
Copy link
Contributor

simi commented Dec 10, 2022

Sad to see this "soft" rejected by Ruby itself (https://bugs.ruby-lang.org/issues/18951).

Would it be possible to make it compatible with ENV as well? That's also often used in tests.

@byroot
Copy link
Member Author

byroot commented Dec 12, 2022

For ENV since that is really only really used in test, an ActiveSupport::TestCase#with_envhelper would do fine (I have one copy pasted in a bunch of apps).

@simi
Copy link
Contributor

simi commented Dec 12, 2022

Would it make sense to wait for https://bugs.ruby-lang.org/issues/18951 to be rejected (closed)? Do you think there is still any chance to unblock that?

@byroot
Copy link
Member Author

byroot commented Dec 12, 2022

I don't think so. I tried several times already.

@simi
Copy link
Contributor

simi commented Dec 12, 2022

For ENV since that is really only really used in test, an ActiveSupport::TestCase#with_envhelper would do fine (I have one copy pasted in a bunch of apps).

🤔 If that's enough in tests and original description is also mostly test oriented, would it make sense to start with with test helper for now instead of introducing global Object#with?

@byroot
Copy link
Member Author

byroot commented Dec 12, 2022

original description is also mostly test oriented

Again, I pointed to test code, but that's not what I want it for.

@simi
Copy link
Contributor

simi commented Dec 12, 2022

original description is also mostly test oriented

Again, I pointed to test code, but that's not what I want it for.

Ahh, sorry. I have missed #46681 (comment). Than the last concern from my side would be the adoption of with method in various gems (mentioned at https://bugs.ruby-lang.org/issues/18951#note-17 for example). Maybe start with less generic name like with_attr/with_attrs?

@byroot
Copy link
Member Author

byroot commented Dec 12, 2022

Than the last concern from my side would be the adoption of with method in various gems

I disagree with that assessment. If you look at the detail of these methods the vast majority is internal APIs, or objects you'd never set attributes on.

I also disagree with the comparison with __send__ and __object__, these methods are things you need to be able to call on arbitrary unknown for metaprogramming reasons. That's not the case of with.

@dhh
Copy link
Member

dhh commented Mar 4, 2023

Ohhh, I just wrote several of those damn ensure clauses for MRSK. Let's make this happen. Respect that matz don't want it in core, but I like both the naming and the placement for Active Support.

Use case

A very common pattern in Ruby, especially in testing is to save the value of an attribute, set a new value, and then restore the old value in an `ensure` clause.

e.g. in unit tests

```ruby
def test_something_when_enabled
  enabled_was, SomeLibrary.enabled = SomeLibrary.enabled, true
  # test things
ensure
  SomeLibrary.enabled = enabled_was
end
```

Or sometime in actual APIs:

```ruby
def with_something_enabled
  enabled_was = @enabled
  @enabled = true
  yield
ensure
  @enabled = enabled_was
end
```

There is no inherent problem with this pattern, but it can be easy to make a mistake, for instance the unit test example:

```ruby
def test_something_when_enabled
  some_call_that_may_raise
  enabled_was, SomeLibrary.enabled = SomeLibrary.enabled, true
  # test things
ensure
  SomeLibrary.enabled = enabled_was
end
```

In the above if `some_call_that_may_raise` actually raises, `SomeLibrary.enabled` is set back to `nil` rather than its original value. I've seen this mistake quite frequently.

Object#with

I think it would be very useful to have a method on Object to implement this pattern in a correct and easy to use way.

NB: `public_send` is used because I don't think such method should be usable if the accessors are private.

With usage:

```ruby
def test_something_when_enabled
  SomeLibrary.with(enabled: true) do
    # test things
  end
end
```

```ruby
GC.with(measure_total_time: true, auto_compact: false) do
  # do something
end
```

Lots of tests in Rails's codebase could be simplified, e.g.:
  - Changing `Thread.report_on_exception`: https://github.com/rails/rails/blob/2d2fdc941e7497ca77f99ce5ad404b6e58f043ef/activerecord/test/cases/connection_pool_test.rb#L583-L595
  - Changing a class attribute: https://github.com/rails/rails/blob/2d2fdc941e7497ca77f99ce5ad404b6e58f043ef/activerecord/test/cases/associations/belongs_to_associations_test.rb#L136-L150
@byroot
Copy link
Member Author

byroot commented Mar 6, 2023

Ok, I rebased and hardened the tests a bit.

@dorianmariecom
Copy link
Contributor

What about exceptions that could be raised and put the object in the intermediate state?

What about thread-safety?

That's what I had to do in climate_control https://github.com/thoughtbot/climate_control

@byroot
Copy link
Member Author

byroot commented Mar 13, 2023

What about exceptions that could be raised and put the object in the intermediate state?

That's tested for: https://github.com/rails/rails/pull/46681/files#diff-07330a39026ef5856cdbc07bfeb738810459904fe6c663207b8dac9b96d97049R61-R73. If you think a case isn't covered, please open a PR.

What about thread-safety?

I don't see what thread safety there is to account for here. If you use Object#with on an object that is shared between threads, you'll have the exact same thread safety issues that if you used the setter.

That's what I had to do in climate_control https://github.com/thoughtbot/climate_control

I assume you mean: thoughtbot/climate_control@d556bf4, but I don't see what you're getting at.

@gsamokovarov
Copy link
Contributor

Should we consider a different name for this core extension method? Something longer like Object.with_attribute? with is a good method name for DSLs, and we do those often in Ruby.

@byroot
Copy link
Member Author

byroot commented Mar 15, 2023

The method being defined in object doesn't prevent you in any way to define another method with the same name for your DSL.

@solnic
Copy link

solnic commented Mar 16, 2023

@byroot adding this to NilClass will cause confusing errors in people's codebases as this method is very common in many libraries. You could at least ensure nil doesn't respond to it as it makes no sense to have that method on nil.

@byroot
Copy link
Member Author

byroot commented Mar 16, 2023

Sure, not a bad idea.

@byroot
Copy link
Member Author

byroot commented Mar 16, 2023

Done in #47688

@joeldrapper
Copy link
Contributor

joeldrapper commented Mar 16, 2023

I strongly recommend renaming this to with_attr (aliased as with_attrs). The attr abbreviation brings to mind the related concepts of attr_reader, atter_writer, etc. and it's less likely to conflict with other methods.

For example, any Data object will accept with with a block and attributes, but the block will be ignored since Data#with returns a copy. It will fail silently and frustratingly for many people.

Additionally, with_attr / with_attrs could raise a helpful ArgumentError if the object is frozen, since you can’t use this technique with frozen objects.

@solnic mentioned, it doesn't make sense for NilClass to respond to with_attr / with_attrs at all — the same is true of TrueClass, FalseClass, Symbol, etc. We could maintain a list of these cases, but they’d also be covered by the frozen? check.

@byroot
Copy link
Member Author

byroot commented Mar 16, 2023

For example, any Data object will accept with with a block and attributes, but the block will be ignored since Data#with returns a copy. It will fail silently and frustratingly for many people.

This is totally irrelevant. Try this without active support loaded:

Data.define(:foo).new(1).with(foo: 2) do
  raise "Oops this block is useless"
end

Additionally, with_attr / with_attrs could raise a helpful ArgumentError

  • No need to rename to do that
  • frozen.with(foo: 1) will raise the same error than frozen.foo = 1 which IMO is perfectly fine.

it doesn't make sense for NilClass ...

Already changed.

@joeldrapper
Copy link
Contributor

This is totally irrelevant. Try this without active support loaded:

No one is going to try that without ActiveSupport loaded. But if they get used to using ActiveSupport’s Object#with because they’re using ActiveSupport and reading ActiveSupport’s documentation, they may try it on a Data object.

@byroot
Copy link
Member Author

byroot commented Mar 16, 2023

they may try it on a Data object.

That's an hypothetical I don't agree with, and there is similar redefinition of Object methods in the stdlib, e.g. UDPSocket#send, etc.

@joeldrapper
Copy link
Contributor

joeldrapper commented Mar 16, 2023

As far as I can tell, UDPSocket isn't typically used as a base class for a specialised class under a different name. Data#with won't look like Data#with since Data is an abstract class. It'll be User#with, Address#with, Event#with, etc.

#
# It can be used on any object as long as both the reader and writer methods
# are public.
def with(**attributes)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tthis method name is super generic. Very easy to create confusion. Maybe call it #with_attributes instead of just a generic #with?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.