Skip to content

Conversation

leaty
Copy link
Contributor

@leaty leaty commented Mar 19, 2020

Database Migration

NO

Description

Adds the ability for Sonarr v3 to remove seeded torrents from rTorrent. For this to work it requires seeded torrents be in a view called "seeded", but it also doesn't break if the user doesn't implement this in their rTorrent config.

Example in rTorrent config (assuming ratio is enabled):

# Close on ratio and set as seeded
view.add = seeded
view.persistent = seeded
method.set = group.seeding.ratio.command, \
    "d.try_close= ; d.ignore_commands.set=1 ; view.set_visible=seeded"

For those who would like time-based seeding as well (irrelevant to this PR):

# Close on time spent seeding and set as seeded (48h)
method.insert.value = ratio.time, 172800
schedule2 = seed_elapsed, 60, 60, \
    ((d.multicall2, seeding, \
        "branch=\"elapsed.greater=(d.timestamp.finished), (ratio.time)\", \
            group.seeding.ratio.command="))

This may be ported to Sonarr v2, but I was unable to build it on my linux machine for some reason so I can't test it.

Todos

  • Tests
  • Documentation

Issues Fixed or Closed by this PR

@markus101
Copy link
Member

Using this same logic can Sonarr set the seed ratio and/or seed time on torrents when they are added? This way it still utilizes the seed settings in indexers.

As-is the functionality is good for removing, but only if the user has gone through the legwork of adding the ratio limit/seed time themselves.

Don’t worry about v2, this isn’t a critical update it’d receive.

@leaty
Copy link
Contributor Author

leaty commented Mar 19, 2020

I'll need to find a way to set ratio/time per-torrent, perhaps by using groups. I'll look into it.

In the meantime I need to push something I missed, currently this PR could potentially remove seeded torrents before they're complete, because I noticed DownloadEventHub only checks CanBeRemoved and makes sure the status isn't Downloading (I assumed it checked Paused as well).

This should be more correct:

item.CanMoveFiles = item.CanBeRemoved =
    torrent.IsSeeded &&
    torrent.IsFinished &&
    !torrent.IsActive;

@leaty
Copy link
Contributor Author

leaty commented Mar 19, 2020

I just realized you don't actually need to set them per-torrent, but instead just set it for sonarr torrents (as a group). I believe below should work in theory, but I have not tested it because I'd like some input. I recommend Solution 2 because it gives rTorrent users more customization, less groups to deal with (there would be a sonarr_ratio_<indexer> group per indexer in Solution 1), as well as less code in Sonarr.

Solution 1

Sonarr sets ratio/size/time (creates a group).

Example of ratio minimum 3.0 up to maximum 4.0 OR until it reaches upload size (to ignore upload size set max and upload to 0):

group.insert_persistent_view = sonarr_ratio
group.sonarr_ratio.enable =
group.sonarr_ratio.min.set = 300
group.sonarr_ratio.max.set = 400
group.sonarr_ratio.upload.set = 200M
view.set_visible = <hash>, sonarr_ratio

Example of seed time:

method.insert.value = group.sonarr_ratio.time, 172800

This solution is a little bit complicated for seed time, because the schedule code requires the group.sonarr_ratio.time to be set, meaning the user would need to set a default value first and then Sonarr would have to change it but if it doesn't already exist the set call Sonarr makes will yield an error.

Solution 2

User configures ratio in rTorrent for Sonarr and simply gives Sonarr the group name (this would void my current PR).

rTorrent user config

# Create group and set ratio/size/time
group.insert_persistent_view = my_sonarr_ratio
group.my_sonarr_ratio.enable =
group.my_sonarr_ratio.min.set = 300
group.my_sonarr_ratio.max.set = 400
group.my_sonarr_ratio.upload.set = 200M
method.insert.value = group.my_sonarr_ratio.time, 172800

# Close on ratio
method.set = group.seeding.ratio.command, \
    "d.try_close= ; d.ignore_commands.set=1"

# Close on time spent seeding (48h)
schedule2 = seed_elapsed, 60, 60, \
    ((d.multicall2, complete, \
        "branch=\"elapsed.greater=(d.timestamp.finished), (group.my_sonarr_ratio.time)\", \
            group.seeding.ratio.command="))

Sonarr when sending torrents to rTorrent

view.set_visible = <hash>, <given_group>

Sonarr determining when to remove

Sonarr can use the following calls to determine this. For using "default" rTorrent ratio use e.g. group2.seeding.ratio.min, but there is no "default" for seed time.

d.complete // download complete
d.is_active // paused/stopped or not
d.ratio // current ratio
d.up.total // size uploaded
d.timestamp.finished // when torrent started seeding

group.<given_group>.ratio.min // minimum ratio to achieve
group.<given_group>.ratio.max // extra ratio when upload size is not yet achieved
group.<given_group>.ratio.upload // max upload size after min ratio
group.<given_group>.time // seed time max (ignores minimum ratio)

@markus101
Copy link
Member

The problem with setting them up for all Sonarr torrents is different indexers have different rules.

Other clients have per torrent limits set when the torrent is added, for rTorrent if it's only possible at the group level there would need to be a group per ratio/time limit combination and some way to figure out which group that is. I think that's what solution 1 achieves, in which case that's the approach we should take. The one thing that isn't clear is whether we'd need to track the group that was used or if we would have to check each group on the torrent to see if the goals have been met.

@leaty
Copy link
Contributor Author

leaty commented Mar 28, 2020

Yes, there would need to be a group per ratio/time (i.e. per indexer). If Sonarr stores which group a torrent is set to (a group setting in Indexer 1), Sonarr can check if the torrent is still in that group using d.views.has.

I understand that rTorrent works very differently in comparison to other clients and therefor it has difficulty with following a uniform pattern. While it is possible to do it properly according to the rTorrent ways, it introduces unique code and especially requiring new settings in Sonarr. But I think there might be a way out after thinking on it a bit.

Solution 3

Sonarr manually handles everything, but in a slightly rTorrent hacky way, yet it retains uniformity with how Sonarr manages other clients.

On new torrent

Create a view for ratio/time if not exists

Sonarr creates a view called e.g. sonarr_{indexer_name}_{ratio}_{time}. Sonarr can check if it already exists using view.list. No need to set ratio/time in rTorrent.

Add torrent to view

view.set_visible = <hash>, sonarr_{indexer_name}_{ratio}_{time}

Check if seeded according to indexer rules

  • Check if torrent has one of the sonarr views with d.views.has. With this you can also determine which ratio/time is set on this torrent, by parsing the view name.
  • Compare d.ratio with indexer setting.
  • Compare d.timestamp.finished with now and check it against indexer setting.

Stop and erase torrent if seeded

d.stop
d.close
d.erase

Of course, you could set the view name to only the indexer name or internal Sonarr indexer ID, parse that and fetch the settings. But if you need a torrent to retain settings even after you change indexer settings, view name has to store ratio/time. However, to avoid creating another view after user changes indexer settings, it's probably easier if view name only holds indexer ID/name, otherwise you'd have to remove the now-invalid sonarr views to not clog rTorrent with garbage views.

@markus101
Copy link
Member

Check if torrent has one of the sonarr views with d.views.has.

Is there a way to get all views back when fetching the torrents? Checking if each torrent has a specific view whenever Sonarr gets the torrents list would be very slow. And checking for each view by a specific indexer name, ratio and time wouldn't work if any of those values change.

Compare d.ratio with indexer setting.
Compare d.timestamp.finished with now and check it against indexer setting.

This is also something that shouldn't be checked per torrent. We intentionally don't want Sonarr to manage whether the ratio has been met by checking it's own settings.

Does d.views give us a list of all views on the torrent?

If so we could add a view (or 2) for each torrent that is added. sonarr_seed_ratio_x and sonarr_seed_time_y, then when we get the torrent and it's views we'd look for those values, parse out the ratio/time limit and compare to the ratio and time the torrent has been seeding. So instead of comparing back to the indexer's settings we'd just compare with what we've already gotten.

It's pretty close to what we do for Transmission since we see it paused, we know the ratio limit (from the torrent or globally) and the current ratio then can decide whether it was paused because it met that limit. The only downside to it not being paused/stopped by rtorrent that jumps to mind is the user couldn't force it to seed longer without removing the views.

@leaty
Copy link
Contributor Author

leaty commented Apr 10, 2020

Is there a way to get all views back when fetching the torrents?

Yes, by using d.multicall2 you can get an array of each torrent with multiple fields. Example using xmlrpc:

xmlrpc http://127.0.0.1:5002 d.multicall2 'array/("")' main \ 
    d.views= \
    d.name= \
    d.hash= \
    d.ratio= \
    d.timestamp.finished=

# Output of the 24th torrent
Index 24 Array of 5 items:
    Index  0 Array of 2 items:
        Index  0 String: 'sonarr_seed_ratio_10'
        Index  1 String: 'sonarr_seed_time_172800'
    Index  1 String: 'ubuntu.iso'
    Index  2 String: 'AAA856593EE916AD606E342A90D8DD9EF3EAAF33'
    Index  3 64-bit integer: 3347
    Index  4 64-bit integer: 1586527636

Does d.views give us a list of all views on the torrent?

Yes and no, d.views provides a list of all non-default views for a given torrent (i.e. set manually or by script), not the default views such as active, started etc. Which is actually better for our intentions anyway (less data).

The only downside to it not being paused/stopped by rtorrent that jumps to mind is the user couldn't force it to seed longer without removing the views.

Removing the sonarr label instead of the views is probably easier. While the solution certainly isn't perfect, it does help the user quite a bit. The manual solution is a bit iffy when it comes to time spent seeding. But it could potentially be an option, i.e. delete when stopped in rtorrent (still checks ratio/time before delete).

@bakerboy448
Copy link
Contributor

@leaty how is this looking?

it'd be a huge feature for those of us on seedboxes

@leaty
Copy link
Contributor Author

leaty commented Sep 13, 2020

@bakerboy448 I've not had the time yet to write a better solution, which in my opinion would be Solution 3. However, the current 2 commits in this pull request with the additional rtorrent configuration of the first comment have been running on my server since Mars and it's still working perfectly for my use case.

@leaty
Copy link
Contributor Author

leaty commented Jan 3, 2021

I've finally had time for this and I managed to implement a similar but slightly better solution than Solution 3. It should be feature complete except for the below drawback.

The only downside to it not being paused/stopped by rtorrent that jumps to mind is the user couldn't force it to seed longer without removing the views.

To solve that issue the user could remove the Sonarr label, as is stated in the ProviderMessage.

@markus101 Can you confirm that a torrent should be removed if it either exceeded ratio or time? I was a little confused about this, but to me it makes the most sense.

@Taloth
Copy link
Member

Taloth commented Jan 3, 2021

Can you confirm that a torrent should be removed if it either exceeded ratio or time? I was a little confused about this, but to me it makes the most sense.

Depends on the tracker normally. BTN for example requires you to seed for a day or till you reached ratio (so OR). But others might say you need to seed for at least a day and reach the ratio.
I think the first approach is most prevalent in torrent clients as it makes the most sense.

@Taloth
Copy link
Member

Taloth commented Jan 3, 2021

As for the whole feature. I wasn't really involved in the whole discussion. But here are some thoughts.

I don't think that using views is a good approach, it feels a bit hacky and it defeats the purpose of delegating seed criteria checks. After all it'd still be Sonarr that does the check and stops the actual seeding.

My concern is two fold:

  1. I rather not have Sonarr itself do the seed criteria checks, because Sonarr isn't a 'seed manager', unless there's absolutely no other way.
  2. We need to avoid something that conflicts with what the user might've set up in their client. Particularly ratio groups. This is my biggest issue with rtorrent since it has strong customization but not that much standardization of common features.

I'm not that concerned with a ton of ratio groups, users will unlikely have more then 4 different seed criteria.

I also think there might be a hybrid approach: Create a ratio group that contains the longest criteria in Sonarr settings. Everything goes in there. Then do the SeedManager check in Sonarr for whatever applies to the individual torrent.
It means if sonarr is gone or the torrents gets another label, then the criteria in the ratio group still kicks in, albeit later.

I'll mull it over some more later.

@leaty
Copy link
Contributor Author

leaty commented Jan 3, 2021

As you say, it is indeed hacky - on the rtorrent side especially, it is however more in line with how Sonarr manages other clients. While the "storage location" for seeding rules here is not the official rtorrent way, i.e. using pure custom views instead of group/group2/custom groups. Although a custom group is still technically a view apparently.

The major difference is that numbers such as ratio.min/ratio.max are stored a little more handsomely, they're stored as values rather than as part of the group/view name, and they are also scriptable in the configuration file naturally. There also isn't (as far as I've investigated in both docs and source) a good way of setting custom groups and scripted options remotely, they seem to be pretty limited for use in the configuration file.

At least in my research I've only seen two ways, you either let the user script rtorrent, or let Sonarr handle a little bit more than it should.

Thanks for the input!

@leaty
Copy link
Contributor Author

leaty commented Jan 3, 2021

Even if it is possible to set this up properly from remote, it's still a little hacky, a group is a group, it's not meant for a singular torrent. It has no per-torrent options for this, of which other clients do. The best compromise I can see would be if it's one group per indexer, but then torrents would not retain their settings.

I also think there might be a hybrid approach: Create a ratio group that contains the longest criteria in Sonarr settings. Everything goes in there. Then do the SeedManager check in Sonarr for whatever applies to the individual torrent.

The hybrid approach is not bad by any means, but technically a group wouldn't really do anything without scripting and telling rtorrent what to do with it, something I believe isn't possible to do remotely? If I'm right then Sonarr would still have to act as the SeedManager and check this group on top of checking the individual torrent.

@Taloth
Copy link
Member

Taloth commented Jan 3, 2021

As you say, it is indeed hacky - on the rtorrent side especially, it is however more in line with how Sonarr manages other clients

Yes, but most other clients have a built-in mechanism to 'do something' with the criteria. Even if it's just pausing the torrent.

I don't like storing the seed criteria in the view name, that's the real hacky part. I do understand the conundrum, there's no apparent solution to it.

There also isn't (as far as I've investigated in both docs and source) a good way of setting custom groups and scripted options remotely, they seem to be pretty limited for use in the configuration file.

Early on we did something for meta downloads to set the sonarr label, it's not needed anymore so it's not in the code, but here was the code we had 2326db0#diff-3a45d8b3fe975df2bfb8c9d9b81dc77774bff042798030b9b1f6a97f95835887L224-L251
I don't think it survives a rtorrent restart, which was why we did away with it for the metadownloads. But I'd be really surprised if you can't use it to add a schedule.
But my argument here is that if the user wants removal, they can add the script to their config. Or we can set up a temporary schedule. Or we can use the group limits to do the check inside Sonarr.
Regardless of the option, Sonarr would set the ratio group settings to make it functional.

@Taloth
Copy link
Member

Taloth commented Jan 3, 2021

Even if it is possible to set this up properly from remote, it's still a little hacky, a group is a group, it's not meant for a singular torrent.

It doesn't have to be. Sonarr will only have a handful of different criteria.

@leaty
Copy link
Contributor Author

leaty commented Jan 3, 2021

I will not deny that storing settings in the view name is ugly, regardless if it's rtorrent, a view, or something entirely else. It's simply ugly to store options within a string. It's important to note however that it doesn't affect rtorrent in any way, a custom view stored on a singular torrent like this is basically a category, a tag or a flag.

Thanks for that code, I will take a look. I do know that by setting a persistent view.. well, it persists, so I wonder if one could somehow persist a schedule, I am quite doubtful though. If not, and assuming I get somewhere, do you think it could be an idea to perhaps not set a persistent view when creating the group - or instead, setting some other form of temporary little variable so that Sonarr will be informed of when to apply the temporary options and scripts again? i.e. after a restart.

@leaty
Copy link
Contributor Author

leaty commented Jan 4, 2021

I've been thinking, perhaps requiring the user to configure isn't so much of an annoyance after all. It is already very customizable and it requires configuration for a lot of basic things. I think mentioning the required configuration as a link to documentation in ProviderMessage is good enough. The reason why I say this is because even if- Sonarr adds scripts to rtorrent for it to manage seeding properly instead of Sonarr managing it, it can easily break user configurations. I only see the current commit the best way for a fully automatic experience, it will most likely never clash with user configurations, but it still removes options from the user as it enforces removal when configured in an indexer.

An rtorrent configuration piece would simplify a lot of things, instead of what's currently being done - which is pretty much "Sonarr configuring rtorrent to help the user". I mean it does work just fine, were it to be merged, but it lacks the ability to continue seeding in a proper way, and as stated it removes customization.

As with any API, you specify the requirements, Sonarr could simply listen for these requirements, and if they are not satisfactory, it will not erase. Similar to the original commits, a decent enough configuration in rtorrent should do; a configuration which may vary, however the torrent has to be paused and set to, perhaps X view or X label, for it to be removed. This would retain the ability for rtorrent to manage seeding fully and completely, but not just that, it would avoid any sort of clash, as it's the user making the necessary configurations.

However this isn't easy either, the remaining problem is per-indexer - or worse even per-torrent options, as one group can only hold one set of parameters for seeding. The best I can see in this instance is per-indexer groups which the user can configure accordingly, but that requires the indexer name- or id as the group name for Sonarr's ability to apply the correct group for the newly added torrent, something I'm not sure where to find, and is probably mildly? annoying for the user, but in the end survivable.

I'll be happy to hear your thoughts on this.

EDIT: I forgot to mention that Sonarr would be updating the defined groups with the indexer seed rules.

@Taloth
Copy link
Member

Taloth commented Jan 4, 2021

I've been thinking, perhaps requiring the user to configure isn't so much of an annoyance after all. It is already very customizable and it requires configuration for a lot of basic things.

I agree. Note that the config is only necessary to perform the removal is Sonarr isn't running.
Optionally we can have an option in rtorrent settings that determines what Sonarr is allowed to do, like removing.

One concern is that if Sonarr sets a ratio group, and the user have their configuration set to remove torrents, then torrents can be removed prior to being imported. (Normally this is dealt with by telling the user to explicitly 'pause' torrents rather than remove them, but it's something that the user would have to configure specifically.)

however the torrent has to be paused and set to, perhaps X view or X label, for it to be removed. This would retain the ability for rtorrent to manage seeding fully and completely, but not just that, it would avoid any sort of clash, as it's the user making the necessary configurations.

Exactly, Sonarr has a MarkItemAsImported function. Currently used for PostImport Category but it can be used for applying a view as well.
Which means the user can script it to handle the scenario. Note however that many users will not be experienced with rtorrent configuration, relying on the default config.
By default for current clients, Sonarr only removes if the seeding criteria have been met. This can apply here as well because either the user or Sonarr would have to apply a ratio group.
I'm not sure about the best approach to handle both the 'novice user' and the 'power user'. Adding the beforementioned config option could be advantageous.

However this isn't easy either, the remaining problem is per-indexer - or worse even per-torrent options, as one group can only hold one set of parameters for seeding.

The trick here is to query the rtorrent api for the groups that are present. And retain an in-memory cache of that.
That way Sonarr can 'deduplicate' the groups based on the criteria. So you don't need per-indexer or per-torrent, just pre-unique-criteria.
One question there is whether groups are persistent on rtorrent restart.

PS: If we set ratio groups, we probably need to check Flood to see if it can be implemented similarly.

@leaty
Copy link
Contributor Author

leaty commented Jan 4, 2021

I'm not sure about the best approach to handle both the 'novice user' and the 'power user'. Adding the beforementioned config option could be advantageous.

I think we're unable to support the novice user here, after all, they copied an existing rtorrent configuration already, so they might as well copy the configuration required for Sonarr's abilities, especially if they want it enough. It should be just a simple name change to apply for their own indexer name.

The trick here is to query the rtorrent api for the groups that are present. And retain an in-memory cache of that.
That way Sonarr can 'deduplicate' the groups based on the criteria. So you don't need per-indexer or per-torrent, just pre-unique-criteria.

I'm not entirely with you on this one, could you ELI5? I understand of course the in-memory piece, but not the rest.

I will comment back with an example configuration to clarify my ideas here.

@Taloth
Copy link
Member

Taloth commented Jan 4, 2021

I think we're unable to support the novice user here

I think you underestimate the amount of users that simply take the default rtorrent docker image. And those with default seedbox config.

I'm not entirely with you on this one

Sonarr has indexer settings for episodes & seasons.
For public trackers, users will often all criteria set similarly.
For private trackers, users will either seed indefinitely, or have tracker specific limits (often similar).

So imagine sonarr grabs a release from indexer x, it'll determine the seed criteria for that indexer. Then it'll check the list of known groups, if there's a group matching those criteria it'll assign the group, otherwise it'll create a new group for those criteria.
In practice this means that there'll be a handful of Sonarr-specific ratio groups.

Ideally it should be possible for Sonarr to determine which groups it created, so it can clear up unused ones.

I will comment back with an example configuration to clarify my ideas here.

Perfect 👍

@leaty
Copy link
Contributor Author

leaty commented Jan 8, 2021

Apologies for the wait, here's what I was thinking.

Mind that this is very similar to Solution 1, however it's at least per-indexer rather than global for all Sonarr torrents, the only thing potentially missing is that already added torrents will not keep the same ratio settings indefinitely, as well as not being super novice-friendly. Also mind that I will mention time-based seeding at the end, as that isn't really natively supported in rtorrent, but rather something you can script together.

rTorrent user config

group.insert_persistent_view = sonarr_{indexer_name_or_id}, sonarr_{indexer_name_or_id}
method.set=group.sonarr_{indexer_name_or_id}.ratio.command,d.try_close=

Replacing indexer_name_or_id with e.g. Nyaa or 5. Although I'm not sure where the user could find the indexer ID. As for the indexer name, of course, rtorrent groups/views can only be alphanumeric and contain underscore, so the indexer name must be so within Sonarr as well.

Sonarr when checking for torrents in rTorrent

group.sonarr_{indexer_name_or_id}.ratio.min.set = <ratio>
group.sonarr_{indexer_name_or_id}.ratio.max.set = 0
group.sonarr_{indexer_name_or_id}.ratio.upload.set = 0
group.sonarr_{indexer_name_or_id}.ratio.enable

While the group is persistent, the ratio settings are not, therefor Sonarr must enable the ratio group rather than the user, otherwise it will be enabled before Sonarr has a chance to set the proper ratio.

If we require the user to set both max and upload to either 0 or what the user wants, then that can be ignored below, but if we do not require it- because the defaults is not 0 we must set them to 0 from Sonarr because neither of which settings exist within Sonarr and thus cannot be updated to any reasonable value.

Note: It's possible the ratio changes could be done upon saving the indexer settings as well, but it doesn't really matter. Also, as you might've guessed by the above commands, this changes the ratio settings for all the torrents already in that group.

Sonarr when removing torrents from rTorrent

Because rtorrent is the SeedManager now, Sonarr only has to check if the torrent is both finished and closed (stopped).

Sonarr when adding torrents to rTorrent

view.set_visible = <hash>, sonarr_<indexer_name_or_id>

@leaty
Copy link
Contributor Author

leaty commented Jan 8, 2021

I also want to mention that I did attempt to add groups remotely via xmlrpc in the terminal for allowing per-torrent settings rather than per-indexer i.e. sonarr_<hash> instead of sonarr_<indexer>, but I was unsuccessful as it's just giving me errors. Even if it can be done, I personally don't love the idea of having 1 group per 1 torrent.

@leaty
Copy link
Contributor Author

leaty commented Jan 8, 2021

I forgot about the time-based seeding, so here goes.

Time based seeding in rTorrent configuration

method.insert.value = group.sonarr_{indexer_name_or_id}.ratio.time, 3153600000

schedule2 = seed_elapsed, 60, 60, \
    ((d.multicall2, sonarr_{indexer_name_or_id}, \
        "branch=\"elapsed.greater=(d.timestamp.finished), (group.sonarr_{indexer_name_or_id}.ratio.time)\", \
            group.sonarr_{indexer_name_or_id}.command="))

Since the time updated by Sonarr below is not persistent, the user should set the default time to something very high, in this example it's a whole hundred years, to avoid premature execution of the ratio command when Sonarr is stopped and rtorrent is restarted.

Sonarr updates time per-indexer when checking for torrents in rTorrent

group.sonarr_{indexer_name_or_id}.ratio.time.set = <time>

@leaty
Copy link
Contributor Author

leaty commented Jan 8, 2021

There may be better solutions than any of this, but my major issue is that I just cannot get methods such as method.set, method.insert.value, schedule2, group.insert_persistent_view, group.insert to work remotely, I mainly test with the xmlrpc command in the terminal, so maybe that is my issue, I just don't know at this time.

I will try some more in the meantime as I'm awaiting feedback.

@leaty
Copy link
Contributor Author

leaty commented Jan 8, 2021

Finally! I got those commands to work, apparently for some commands you need an empty target parameter as the first argument when run remotely rather than when it's inside the configuration file. It's pretty much like this:

group.insert_persistent_view = "", nyaa, nyaa

As for the xmlrpc command in the terminal that I was using, it requires the same but you also need to specify types, but in only those cases, e.g.

# Create custom property
xmlrpc localhost:5001 method.insert.value s/ group.nyaa.ratio.time i/1337

# Update custom property
xmlrpc localhost:5001 group.nyaa.ratio.time.set s/ 1337

# Update standard property
xmlrpc localhost:5001 group.nyaa.ratio.min.set 1337

That's pretty confusing as to why it is like this, perhaps someone could enlighten me here. But regardless, I'm now able to create groups remotely- and the rest, so I'm very keen on hearing that final perfect idea that matches both rtorrent's and Sonarr's principles before I get started, e.g. per-indexer vs per-torrent vs unique. @Taloth

The only remaining issue I can see is custom configuration overriding Sonarr, i.e. the default group.seeding.ratio.min or simply ratio.min the user could have configured might take priority if the ratio is lower, I'll have to test I suppose, but it might not be entirely wrong that it would either.

@Taloth
Copy link
Member

Taloth commented Jan 8, 2021

That's pretty confusing as to why it is like this, perhaps someone could enlighten me here

yeah.... I had similar trouble when I did the scripting stuff years ago. The api has targets, the config file can't because it's by definition global. So I suspect that's the reason for the target being absent in the config. However, 50-50 I'm totally wrong about that.

While the group is persistent, the ratio settings are not

Wait, WHAT?! Why not? That kinda defeats the purpose of storing the seed criteria in the download client does it not?
When editing ratio group limits in rutorrent they are persistent. Are ratio groups different from groups?

@leaty
Copy link
Contributor Author

leaty commented Jan 10, 2021

The api has targets, the config file can't because it's by definition global.

It's a good enough theory to believe, assuming then that the empty target means global- one would think you could supply a different target, and apparently by trial and error you can use a torrent hash as a target when e.g. adding a group, but it doesn't really... do anything different, and it's not persistent either as one would have thought, as if it's "stored" in a torrent, but it really doesn't seem to be stored on the torrent at all, it's still global. For other functions such as view.set_visible, you normally use the hash as the first argument, so there it at least makes sense.

Wait, WHAT?! Why not? That kinda defeats the purpose of storing the seed criteria in the download client does it not?

Lol, yeah. What I really mean is that the group itself, with all its settings does not persist, but the view set on torrents do, so when the group is re-added again with the same view name, the torrent is again linked to the group and its settings, but the settings also have to be set again as they are lost and instead set to defaults such as ratio.min 200 and ratio.max 300.

I really believe you're not really supposed to be "modding" rtorrent like this remotely, but rather it relies on the configuration upon restart to remain the same and with that you have true persistency. I will demonstrate below:

First I'll add a view and a group both named mygroup, enable the ratio, set it to 1337 and add it to a torrent.

leaty@arch:~$ xmlrpc localhost:5001 group.insert_persistent_view s/ mygroup mygroup
leaty@arch:~$ xmlrpc localhost:5001 group.mygroup.ratio.enable
leaty@arch:~$ xmlrpc localhost:5001 group.mygroup.ratio.min.set 1337
leaty@arch:~$ xmlrpc localhost:5001 view.set_visible 5FFF0E1C8AC414860310BCC1CB76AC28E960EFBE mygroup

I'll check for commands relating to mygroup and check the ratio.

leaty@arch:~$ xmlrpc localhost:5001 system.listMethods | grep mygroup
  Index 738 String: 'group.mygroup.ratio.enable'
  Index 739 String: 'group.mygroup.ratio.disable'
  Index 740 String: 'group.mygroup.ratio.command'
  Index 741 String: 'group2.mygroup.view'
  Index 742 String: 'group2.mygroup.view.set'
  Index 743 String: 'group2.mygroup.ratio.min'
  Index 744 String: 'group2.mygroup.ratio.min.set'
  Index 745 String: 'group2.mygroup.ratio.max'
  Index 746 String: 'group2.mygroup.ratio.max.set'
  Index 747 String: 'group2.mygroup.ratio.upload'
  Index 748 String: 'group2.mygroup.ratio.upload.set'
  Index 749 String: 'group.mygroup.view'
  Index 750 String: 'group.mygroup.view.set'
  Index 751 String: 'group.mygroup.ratio.min'
  Index 752 String: 'group.mygroup.ratio.min.set'
  Index 753 String: 'group.mygroup.ratio.max'
  Index 754 String: 'group.mygroup.ratio.max.set'
  Index 755 String: 'group.mygroup.ratio.upload'
  Index 756 String: 'group.mygroup.ratio.upload.set'
leaty@arch:~$ xmlrpc localhost:5001 group.mygroup.ratio.min
Result:

64-bit integer: 1337

Next, after restarting rtorrent.

leaty@arch:~$ xmlrpc localhost:5001 system.listMethods | grep mygroup
leaty@arch:~$ xmlrpc localhost:5001 group.mygroup.ratio.min
Failed.  Call failed.  RPC failed at server.  Method 'group.mygroup.ratio.min' not defined.  (XML-RPC fault code -506)

The view is still set on the torrent though.

leaty@arch:~$ xmlrpc localhost:5001 d.multicall2 'array/("")' main d.name= d.hash= d.views=
Result:

Array of 1 items:
  Index  0 Array of 3 items:
             Index  0 String: 'ubuntu-20.10-desktop-amd64.iso'
             Index  1 String: '5FFF0E1C8AC414860310BCC1CB76AC28E960EFBE'
             Index  2 Array of 1 items:
                        Index  0 String: 'mygroup'

After re-adding the group, the settings are reset.

leaty@arch:~$ xmlrpc localhost:5001 group.insert_persistent_view s/ mygroup mygroup
leaty@arch:~$ xmlrpc localhost:5001 group.mygroup.ratio.min
Result:

64-bit integer: 200

A little extra proof of the group being completely gone after restart- you can't add the same group again.

leaty@arch:~$ xmlrpc localhost:5001 group.insert_persistent_view s/ mygroup mygroup
Failed.  Call failed.  RPC failed at server.  View with same name already inserted..  (XML-RPC fault code -503)

When editing ratio group limits in rutorrent they are persistent. Are ratio groups different from groups?

I can't really speak on rutorrent as I've never used it, but I imagine they do some under the hood magic for this, or simply reapply all groups and settings upon restart/reconnect of rtorrent.

@Taloth
Copy link
Member

Taloth commented Jan 12, 2021

It would be rather user-unfriendly. Perhaps worse than option 5. That's the problem with this whole thing...

@leaty
Copy link
Contributor Author

leaty commented Jan 12, 2021

I don't disagree, just gotta find and pick the lesser evil at this point 😄

@Taloth
Copy link
Member

Taloth commented Jan 12, 2021

And this basically leaves option 3 as most user-friendly. I can do the legwork here in c# world.

I mentioned this a couple of posts ago:

In any case, maybe add a feature that adds a persistent sonarr_imported view to the torrent in MarkAsImported, allowing the user to customize.

Do you think this would be useful for advanced users? I'm also open to adding sonarr_indexer_{id} as well.

@leaty
Copy link
Contributor Author

leaty commented Jan 12, 2021

That's a very good idea, it's a simple but certainly very powerful feature for advanced users. sonarr_indexer_{id} is the cherry on top, but can the user easily determine the id of each indexer?

@Taloth
Copy link
Member

Taloth commented Jan 12, 2021

but can the user easily determine the id of each indexer?

Easy 'enough', i'd say, it's not visible in the UI, but readily available on the api. For any advanced user it should be trivial.
Benefit of indexerId over anything else is that it's static, doesn't need another field somewhere, and is usable for advanced users.

The standard users have a convenient seed criteria. While advanced users have a method of preventing sonarr from removing it (by not enabling Remove) and still have enough info to do it themselves. Yeah, I kinda like it.
I suppose the only thing that's tricky for advanced user to determine is whether it's a season pack.

@leaty
Copy link
Contributor Author

leaty commented Jan 12, 2021

Agreed then, include the cherry for sure.

I suppose the only thing that's tricky for advanced user to determine is whether it's a season pack.

This just gave me flashbacks, when I started this journey the first thing I tried was Sonarr's Connect -> Custom Script, my script was deleting the whole torrent and the torrent directory, I thought I was a genius, then the season pack came.. It managed 1 import out of 50, then it was gone.

Do you think, perhaps as a blueberry on top, storing a view for season packs is a good idea? Or is the blueberry overkill. Yes, sorry I got the sudden urge for berries.

@Taloth
Copy link
Member

Taloth commented Jan 12, 2021

Do you think, perhaps as a blueberry on top, storing a view for season packs is a good idea?

You don't need that for removal, you only need that if you want to throw it in another ratio group since trackers like BTN have different criteria for packs. But sure.

Also, now I think of it, do we actually need sonarr_imported view or is the Post Import Category enough?

@leaty
Copy link
Contributor Author

leaty commented Jan 12, 2021

That would be the d.custom1, aka label right? Scripting makes common use of d.multicall2 where the first argument is a view, so e.g. you could use it to set a schedule2 for torrents having sonarr_indexer_{id} or sonarr_imported.

But since I just tried this to make sure- apparently regardless of what I do, nothing is persistent except on torrents.. Yes, sonarr_imported is set as a "view" on the torrent, but the "view" doesn't exist globally anymore after a restart, and thus d.multicall2 fails with "Could not find view" after said restart. I tried view.add, view.persistent and group.insert_persistent_view for good measure.

Scripting will be slightly unorthodox because of that, but it's still useful as there are ways around it. Outside of the configuration you could always call the rtorrent api, grab all the torrents, and decide on an action depending on if d.views.has is 1, or if fetching all the views- then if it exists in the list. Or perhaps there is an easy way to script this as well in the configuration, but I've not done it before.

@leaty
Copy link
Contributor Author

leaty commented Jan 12, 2021

I suppose the advanced users could simply create the same views that Sonarr applies on torrents in the configuration, to ensure persistency and availability of d.multicall2 for scripting purposes, it solves my entire previous post and it still isn't half bad.

So what I mean is, I think d.custom1 loses out on the ease of d.multicall2.

@leaty
Copy link
Contributor Author

leaty commented Jan 13, 2021

@Taloth Does the cache for option 3 survive a sonarr reboot?

@Taloth
Copy link
Member

Taloth commented Jan 13, 2021

The cache gets rebuilt automatically using the DownloadHistory and Indexer settings.

@markus101 markus101 changed the base branch from phantom-develop to develop March 13, 2021 18:29
@markus101 markus101 force-pushed the develop branch 2 times, most recently from c10c3c6 to 1e99856 Compare March 29, 2021 20:28
@leaty leaty force-pushed the phantom-feature-rtorrent-erase branch from 0fedb16 to b4a4425 Compare April 17, 2021 17:24
@leaty
Copy link
Contributor Author

leaty commented Apr 17, 2021

Revamped to last agreed solution, i.e. letting Sonarr handle everything using an internal cache. Also sets a post-import view for those who want some rtorrent scripting support. Most of the code is from @Taloth, see here: 6143d20. Huge thanks for this nearly working example!

Let me know if something should be changed.

@leaty leaty force-pushed the phantom-feature-rtorrent-erase branch from 53a6bc0 to 81aab22 Compare April 17, 2021 17:40
Copy link
Member

@Taloth Taloth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll check if qbit still works and see what it takes to move CDH Remove flag and deal with the migration.

@@ -108,6 +113,7 @@ public List<RTorrentTorrent> GetTorrents(RTorrentSettings settings)
item.IsOpen = Convert.ToBoolean((long)torrent[8]);
item.IsActive = Convert.ToBoolean((long)torrent[9]);
item.IsFinished = Convert.ToBoolean((long)torrent[10]);
item.FinishedTime = (long)torrent[11];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What value is FinishedTime if it's still being downloaded?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be 0, or more specifically 64-bit integer: 0.

@leaty leaty force-pushed the phantom-feature-rtorrent-erase branch from beeb071 to 7ae8bcd Compare May 15, 2021 22:38
@leaty leaty force-pushed the phantom-feature-rtorrent-erase branch from 7ae8bcd to d93923f Compare May 15, 2021 22:47
@Taloth Taloth merged commit 91fe47e into Sonarr:develop May 15, 2021
@pyroscope
Copy link

FYI, the proper place to store per-item persistent settings is named customs, e.g. a sonarr_delete_by datestamp.

A good and common way to protect items against automation is either the "Ignore commands" flag, or setting "prio=3" (and then using that to filter them out).

ruT gets some things "magically" reconfigured by requiring you to call their "init" script as part of the startup configuration.

@leaty
Copy link
Contributor Author

leaty commented May 17, 2021

Good to know thanks, and yeah I assumed ruT worked something like that.

Purely for historic reasons, I'll leave the previous revisions of this PR before it was squashed here.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants