-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Add rtorrent erase feature #3632
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Using this same logic can Sonarr set the seed ratio and/or seed time on torrents when they are added? This way it still utilizes the seed settings in indexers. As-is the functionality is good for removing, but only if the user has gone through the legwork of adding the ratio limit/seed time themselves. Don’t worry about v2, this isn’t a critical update it’d receive. |
I'll need to find a way to set ratio/time per-torrent, perhaps by using groups. I'll look into it. In the meantime I need to push something I missed, currently this PR could potentially remove seeded torrents before they're complete, because I noticed This should be more correct: item.CanMoveFiles = item.CanBeRemoved =
torrent.IsSeeded &&
torrent.IsFinished &&
!torrent.IsActive; |
I just realized you don't actually need to set them per-torrent, but instead just set it for sonarr torrents (as a group). I believe below should work in theory, but I have not tested it because I'd like some input. I recommend Solution 2 because it gives rTorrent users more customization, less groups to deal with (there would be a Solution 1Sonarr sets ratio/size/time (creates a group). Example of ratio minimum 3.0 up to maximum 4.0 OR until it reaches upload size (to ignore upload size set
|
The problem with setting them up for all Sonarr torrents is different indexers have different rules. Other clients have per torrent limits set when the torrent is added, for rTorrent if it's only possible at the group level there would need to be a group per ratio/time limit combination and some way to figure out which group that is. I think that's what solution 1 achieves, in which case that's the approach we should take. The one thing that isn't clear is whether we'd need to track the group that was used or if we would have to check each group on the torrent to see if the goals have been met. |
Yes, there would need to be a group per ratio/time (i.e. per indexer). If Sonarr stores which group a torrent is set to (a group setting in Indexer 1), Sonarr can check if the torrent is still in that group using I understand that rTorrent works very differently in comparison to other clients and therefor it has difficulty with following a uniform pattern. While it is possible to do it properly according to the rTorrent ways, it introduces unique code and especially requiring new settings in Sonarr. But I think there might be a way out after thinking on it a bit. Solution 3Sonarr manually handles everything, but in a slightly rTorrent hacky way, yet it retains uniformity with how Sonarr manages other clients. On new torrentCreate a view for ratio/time if not existsSonarr creates a view called e.g. Add torrent to view
Check if seeded according to indexer rules
Stop and erase torrent if seeded
Of course, you could set the view name to only the indexer name or internal Sonarr indexer ID, parse that and fetch the settings. But if you need a torrent to retain settings even after you change indexer settings, view name has to store ratio/time. However, to avoid creating another view after user changes indexer settings, it's probably easier if view name only holds indexer ID/name, otherwise you'd have to remove the now-invalid sonarr views to not clog rTorrent with garbage views. |
Is there a way to get all views back when fetching the torrents? Checking if each torrent has a specific view whenever Sonarr gets the torrents list would be very slow. And checking for each view by a specific indexer name, ratio and time wouldn't work if any of those values change.
This is also something that shouldn't be checked per torrent. We intentionally don't want Sonarr to manage whether the ratio has been met by checking it's own settings. Does If so we could add a view (or 2) for each torrent that is added. It's pretty close to what we do for Transmission since we see it paused, we know the ratio limit (from the torrent or globally) and the current ratio then can decide whether it was paused because it met that limit. The only downside to it not being paused/stopped by rtorrent that jumps to mind is the user couldn't force it to seed longer without removing the views. |
Yes, by using xmlrpc http://127.0.0.1:5002 d.multicall2 'array/("")' main \
d.views= \
d.name= \
d.hash= \
d.ratio= \
d.timestamp.finished=
# Output of the 24th torrent
Index 24 Array of 5 items:
Index 0 Array of 2 items:
Index 0 String: 'sonarr_seed_ratio_10'
Index 1 String: 'sonarr_seed_time_172800'
Index 1 String: 'ubuntu.iso'
Index 2 String: 'AAA856593EE916AD606E342A90D8DD9EF3EAAF33'
Index 3 64-bit integer: 3347
Index 4 64-bit integer: 1586527636
Yes and no,
Removing the sonarr label instead of the views is probably easier. While the solution certainly isn't perfect, it does help the user quite a bit. The manual solution is a bit iffy when it comes to time spent seeding. But it could potentially be an option, i.e. delete when stopped in rtorrent (still checks ratio/time before delete). |
@leaty how is this looking? it'd be a huge feature for those of us on seedboxes |
@bakerboy448 I've not had the time yet to write a better solution, which in my opinion would be Solution 3. However, the current 2 commits in this pull request with the additional rtorrent configuration of the first comment have been running on my server since Mars and it's still working perfectly for my use case. |
I've finally had time for this and I managed to implement a similar but slightly better solution than Solution 3. It should be feature complete except for the below drawback.
To solve that issue the user could remove the Sonarr label, as is stated in the @markus101 Can you confirm that a torrent should be removed if it either exceeded ratio or time? I was a little confused about this, but to me it makes the most sense. |
Depends on the tracker normally. BTN for example requires you to seed for a day or till you reached ratio (so OR). But others might say you need to seed for at least a day and reach the ratio. |
As for the whole feature. I wasn't really involved in the whole discussion. But here are some thoughts. I don't think that using views is a good approach, it feels a bit hacky and it defeats the purpose of delegating seed criteria checks. After all it'd still be Sonarr that does the check and stops the actual seeding. My concern is two fold:
I'm not that concerned with a ton of ratio groups, users will unlikely have more then 4 different seed criteria. I also think there might be a hybrid approach: Create a ratio group that contains the longest criteria in Sonarr settings. Everything goes in there. Then do the SeedManager check in Sonarr for whatever applies to the individual torrent. I'll mull it over some more later. |
As you say, it is indeed hacky - on the rtorrent side especially, it is however more in line with how Sonarr manages other clients. While the "storage location" for seeding rules here is not the official rtorrent way, i.e. using pure custom views instead of group/group2/custom groups. Although a custom group is still technically a view apparently. The major difference is that numbers such as ratio.min/ratio.max are stored a little more handsomely, they're stored as values rather than as part of the group/view name, and they are also scriptable in the configuration file naturally. There also isn't (as far as I've investigated in both docs and source) a good way of setting custom groups and scripted options remotely, they seem to be pretty limited for use in the configuration file. At least in my research I've only seen two ways, you either let the user script rtorrent, or let Sonarr handle a little bit more than it should. Thanks for the input! |
Even if it is possible to set this up properly from remote, it's still a little hacky, a group is a group, it's not meant for a singular torrent. It has no per-torrent options for this, of which other clients do. The best compromise I can see would be if it's one group per indexer, but then torrents would not retain their settings.
The hybrid approach is not bad by any means, but technically a group wouldn't really do anything without scripting and telling rtorrent what to do with it, something I believe isn't possible to do remotely? If I'm right then Sonarr would still have to act as the SeedManager and check this group on top of checking the individual torrent. |
Yes, but most other clients have a built-in mechanism to 'do something' with the criteria. Even if it's just pausing the torrent. I don't like storing the seed criteria in the view name, that's the real hacky part. I do understand the conundrum, there's no apparent solution to it.
Early on we did something for meta downloads to set the sonarr label, it's not needed anymore so it's not in the code, but here was the code we had 2326db0#diff-3a45d8b3fe975df2bfb8c9d9b81dc77774bff042798030b9b1f6a97f95835887L224-L251 |
It doesn't have to be. Sonarr will only have a handful of different criteria. |
I will not deny that storing settings in the view name is ugly, regardless if it's rtorrent, a view, or something entirely else. It's simply ugly to store options within a string. It's important to note however that it doesn't affect rtorrent in any way, a custom view stored on a singular torrent like this is basically a category, a tag or a flag. Thanks for that code, I will take a look. I do know that by setting a persistent view.. well, it persists, so I wonder if one could somehow persist a schedule, I am quite doubtful though. If not, and assuming I get somewhere, do you think it could be an idea to perhaps not set a persistent view when creating the group - or instead, setting some other form of temporary little variable so that Sonarr will be informed of when to apply the temporary options and scripts again? i.e. after a restart. |
I've been thinking, perhaps requiring the user to configure isn't so much of an annoyance after all. It is already very customizable and it requires configuration for a lot of basic things. I think mentioning the required configuration as a link to documentation in ProviderMessage is good enough. The reason why I say this is because even if- Sonarr adds scripts to rtorrent for it to manage seeding properly instead of Sonarr managing it, it can easily break user configurations. I only see the current commit the best way for a fully automatic experience, it will most likely never clash with user configurations, but it still removes options from the user as it enforces removal when configured in an indexer. An rtorrent configuration piece would simplify a lot of things, instead of what's currently being done - which is pretty much "Sonarr configuring rtorrent to help the user". I mean it does work just fine, were it to be merged, but it lacks the ability to continue seeding in a proper way, and as stated it removes customization. As with any API, you specify the requirements, Sonarr could simply listen for these requirements, and if they are not satisfactory, it will not erase. Similar to the original commits, a decent enough configuration in rtorrent should do; a configuration which may vary, however the torrent has to be paused and set to, perhaps X view or X label, for it to be removed. This would retain the ability for rtorrent to manage seeding fully and completely, but not just that, it would avoid any sort of clash, as it's the user making the necessary configurations. However this isn't easy either, the remaining problem is per-indexer - or worse even per-torrent options, as one group can only hold one set of parameters for seeding. The best I can see in this instance is per-indexer groups which the user can configure accordingly, but that requires the indexer name- or id as the group name for Sonarr's ability to apply the correct group for the newly added torrent, something I'm not sure where to find, and is probably mildly? annoying for the user, but in the end survivable. I'll be happy to hear your thoughts on this. EDIT: I forgot to mention that Sonarr would be updating the defined groups with the indexer seed rules. |
I agree. Note that the config is only necessary to perform the removal is Sonarr isn't running. One concern is that if Sonarr sets a ratio group, and the user have their configuration set to remove torrents, then torrents can be removed prior to being imported. (Normally this is dealt with by telling the user to explicitly 'pause' torrents rather than remove them, but it's something that the user would have to configure specifically.)
Exactly, Sonarr has a MarkItemAsImported function. Currently used for PostImport Category but it can be used for applying a view as well.
The trick here is to query the rtorrent api for the groups that are present. And retain an in-memory cache of that. PS: If we set ratio groups, we probably need to check Flood to see if it can be implemented similarly. |
I think we're unable to support the novice user here, after all, they copied an existing rtorrent configuration already, so they might as well copy the configuration required for Sonarr's abilities, especially if they want it enough. It should be just a simple name change to apply for their own indexer name.
I'm not entirely with you on this one, could you ELI5? I understand of course the in-memory piece, but not the rest. I will comment back with an example configuration to clarify my ideas here. |
I think you underestimate the amount of users that simply take the default rtorrent docker image. And those with default seedbox config.
Sonarr has indexer settings for episodes & seasons. So imagine sonarr grabs a release from indexer x, it'll determine the seed criteria for that indexer. Then it'll check the list of known groups, if there's a group matching those criteria it'll assign the group, otherwise it'll create a new group for those criteria. Ideally it should be possible for Sonarr to determine which groups it created, so it can clear up unused ones.
Perfect 👍 |
Apologies for the wait, here's what I was thinking. Mind that this is very similar to Solution 1, however it's at least per-indexer rather than global for all Sonarr torrents, the only thing potentially missing is that already added torrents will not keep the same ratio settings indefinitely, as well as not being super novice-friendly. Also mind that I will mention time-based seeding at the end, as that isn't really natively supported in rtorrent, but rather something you can script together. rTorrent user configgroup.insert_persistent_view = sonarr_{indexer_name_or_id}, sonarr_{indexer_name_or_id}
method.set=group.sonarr_{indexer_name_or_id}.ratio.command,d.try_close= Replacing Sonarr when checking for torrents in rTorrentgroup.sonarr_{indexer_name_or_id}.ratio.min.set = <ratio>
group.sonarr_{indexer_name_or_id}.ratio.max.set = 0
group.sonarr_{indexer_name_or_id}.ratio.upload.set = 0
group.sonarr_{indexer_name_or_id}.ratio.enable While the group is persistent, the ratio settings are not, therefor Sonarr must enable the ratio group rather than the user, otherwise it will be enabled before Sonarr has a chance to set the proper ratio. If we require the user to set both Note: It's possible the ratio changes could be done upon saving the indexer settings as well, but it doesn't really matter. Also, as you might've guessed by the above commands, this changes the ratio settings for all the torrents already in that group. Sonarr when removing torrents from rTorrentBecause rtorrent is the SeedManager now, Sonarr only has to check if the torrent is both finished and closed (stopped). Sonarr when adding torrents to rTorrentview.set_visible = <hash>, sonarr_<indexer_name_or_id> |
I also want to mention that I did attempt to add groups remotely via xmlrpc in the terminal for allowing per-torrent settings rather than per-indexer i.e. |
I forgot about the time-based seeding, so here goes. Time based seeding in rTorrent configurationmethod.insert.value = group.sonarr_{indexer_name_or_id}.ratio.time, 3153600000
schedule2 = seed_elapsed, 60, 60, \
((d.multicall2, sonarr_{indexer_name_or_id}, \
"branch=\"elapsed.greater=(d.timestamp.finished), (group.sonarr_{indexer_name_or_id}.ratio.time)\", \
group.sonarr_{indexer_name_or_id}.command=")) Since the time updated by Sonarr below is not persistent, the user should set the default time to something very high, in this example it's a whole hundred years, to avoid premature execution of the ratio command when Sonarr is stopped and rtorrent is restarted. Sonarr updates time per-indexer when checking for torrents in rTorrentgroup.sonarr_{indexer_name_or_id}.ratio.time.set = <time> |
There may be better solutions than any of this, but my major issue is that I just cannot get methods such as I will try some more in the meantime as I'm awaiting feedback. |
Finally! I got those commands to work, apparently for some commands you need an empty target parameter as the first argument when run remotely rather than when it's inside the configuration file. It's pretty much like this: group.insert_persistent_view = "", nyaa, nyaa As for the # Create custom property
xmlrpc localhost:5001 method.insert.value s/ group.nyaa.ratio.time i/1337
# Update custom property
xmlrpc localhost:5001 group.nyaa.ratio.time.set s/ 1337
# Update standard property
xmlrpc localhost:5001 group.nyaa.ratio.min.set 1337 That's pretty confusing as to why it is like this, perhaps someone could enlighten me here. But regardless, I'm now able to create groups remotely- and the rest, so I'm very keen on hearing that final perfect idea that matches both rtorrent's and Sonarr's principles before I get started, e.g. per-indexer vs per-torrent vs unique. @Taloth The only remaining issue I can see is custom configuration overriding Sonarr, i.e. the default |
yeah.... I had similar trouble when I did the scripting stuff years ago. The api has targets, the config file can't because it's by definition global. So I suspect that's the reason for the target being absent in the config. However, 50-50 I'm totally wrong about that.
Wait, WHAT?! Why not? That kinda defeats the purpose of storing the seed criteria in the download client does it not? |
It's a good enough theory to believe, assuming then that the empty target means global- one would think you could supply a different target, and apparently by trial and error you can use a torrent hash as a target when e.g. adding a group, but it doesn't really... do anything different, and it's not persistent either as one would have thought, as if it's "stored" in a torrent, but it really doesn't seem to be stored on the torrent at all, it's still global. For other functions such as
Lol, yeah. What I really mean is that the group itself, with all its settings does not persist, but the view set on torrents do, so when the group is re-added again with the same view name, the torrent is again linked to the group and its settings, but the settings also have to be set again as they are lost and instead set to defaults such as ratio.min I really believe you're not really supposed to be "modding" rtorrent like this remotely, but rather it relies on the configuration upon restart to remain the same and with that you have true persistency. I will demonstrate below: First I'll add a view and a group both named leaty@arch:~$ xmlrpc localhost:5001 group.insert_persistent_view s/ mygroup mygroup
leaty@arch:~$ xmlrpc localhost:5001 group.mygroup.ratio.enable
leaty@arch:~$ xmlrpc localhost:5001 group.mygroup.ratio.min.set 1337
leaty@arch:~$ xmlrpc localhost:5001 view.set_visible 5FFF0E1C8AC414860310BCC1CB76AC28E960EFBE mygroup I'll check for commands relating to leaty@arch:~$ xmlrpc localhost:5001 system.listMethods | grep mygroup
Index 738 String: 'group.mygroup.ratio.enable'
Index 739 String: 'group.mygroup.ratio.disable'
Index 740 String: 'group.mygroup.ratio.command'
Index 741 String: 'group2.mygroup.view'
Index 742 String: 'group2.mygroup.view.set'
Index 743 String: 'group2.mygroup.ratio.min'
Index 744 String: 'group2.mygroup.ratio.min.set'
Index 745 String: 'group2.mygroup.ratio.max'
Index 746 String: 'group2.mygroup.ratio.max.set'
Index 747 String: 'group2.mygroup.ratio.upload'
Index 748 String: 'group2.mygroup.ratio.upload.set'
Index 749 String: 'group.mygroup.view'
Index 750 String: 'group.mygroup.view.set'
Index 751 String: 'group.mygroup.ratio.min'
Index 752 String: 'group.mygroup.ratio.min.set'
Index 753 String: 'group.mygroup.ratio.max'
Index 754 String: 'group.mygroup.ratio.max.set'
Index 755 String: 'group.mygroup.ratio.upload'
Index 756 String: 'group.mygroup.ratio.upload.set'
leaty@arch:~$ xmlrpc localhost:5001 group.mygroup.ratio.min
Result:
64-bit integer: 1337 Next, after restarting rtorrent. leaty@arch:~$ xmlrpc localhost:5001 system.listMethods | grep mygroup
leaty@arch:~$ xmlrpc localhost:5001 group.mygroup.ratio.min
Failed. Call failed. RPC failed at server. Method 'group.mygroup.ratio.min' not defined. (XML-RPC fault code -506) The view is still set on the torrent though. leaty@arch:~$ xmlrpc localhost:5001 d.multicall2 'array/("")' main d.name= d.hash= d.views=
Result:
Array of 1 items:
Index 0 Array of 3 items:
Index 0 String: 'ubuntu-20.10-desktop-amd64.iso'
Index 1 String: '5FFF0E1C8AC414860310BCC1CB76AC28E960EFBE'
Index 2 Array of 1 items:
Index 0 String: 'mygroup' After re-adding the group, the settings are reset. leaty@arch:~$ xmlrpc localhost:5001 group.insert_persistent_view s/ mygroup mygroup
leaty@arch:~$ xmlrpc localhost:5001 group.mygroup.ratio.min
Result:
64-bit integer: 200 A little extra proof of the group being completely gone after restart- you can't add the same group again. leaty@arch:~$ xmlrpc localhost:5001 group.insert_persistent_view s/ mygroup mygroup
Failed. Call failed. RPC failed at server. View with same name already inserted.. (XML-RPC fault code -503)
I can't really speak on rutorrent as I've never used it, but I imagine they do some under the hood magic for this, or simply reapply all groups and settings upon restart/reconnect of rtorrent. |
It would be rather user-unfriendly. Perhaps worse than option 5. That's the problem with this whole thing... |
I don't disagree, just gotta find and pick the lesser evil at this point 😄 |
And this basically leaves option 3 as most user-friendly. I can do the legwork here in c# world. I mentioned this a couple of posts ago:
Do you think this would be useful for advanced users? I'm also open to adding sonarr_indexer_{id} as well. |
That's a very good idea, it's a simple but certainly very powerful feature for advanced users. |
Easy 'enough', i'd say, it's not visible in the UI, but readily available on the api. For any advanced user it should be trivial. The standard users have a convenient seed criteria. While advanced users have a method of preventing sonarr from removing it (by not enabling Remove) and still have enough info to do it themselves. Yeah, I kinda like it. |
Agreed then, include the cherry for sure.
This just gave me flashbacks, when I started this journey the first thing I tried was Sonarr's Do you think, perhaps as a blueberry on top, storing a view for season packs is a good idea? Or is the blueberry overkill. Yes, sorry I got the sudden urge for berries. |
You don't need that for removal, you only need that if you want to throw it in another ratio group since trackers like BTN have different criteria for packs. But sure. Also, now I think of it, do we actually need sonarr_imported view or is the Post Import Category enough? |
That would be the But since I just tried this to make sure- apparently regardless of what I do, nothing is persistent except on torrents.. Yes, Scripting will be slightly unorthodox because of that, but it's still useful as there are ways around it. Outside of the configuration you could always call the rtorrent api, grab all the torrents, and decide on an action depending on if |
I suppose the advanced users could simply create the same views that Sonarr applies on torrents in the configuration, to ensure persistency and availability of So what I mean is, I think |
@Taloth Does the cache for option 3 survive a sonarr reboot? |
The cache gets rebuilt automatically using the DownloadHistory and Indexer settings. |
c10c3c6
to
1e99856
Compare
0fedb16
to
b4a4425
Compare
Revamped to last agreed solution, i.e. letting Sonarr handle everything using an internal cache. Also sets a post-import view for those who want some rtorrent scripting support. Most of the code is from @Taloth, see here: 6143d20. Huge thanks for this nearly working example! Let me know if something should be changed. |
53a6bc0
to
81aab22
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll check if qbit still works and see what it takes to move CDH Remove flag and deal with the migration.
@@ -108,6 +113,7 @@ public List<RTorrentTorrent> GetTorrents(RTorrentSettings settings) | |||
item.IsOpen = Convert.ToBoolean((long)torrent[8]); | |||
item.IsActive = Convert.ToBoolean((long)torrent[9]); | |||
item.IsFinished = Convert.ToBoolean((long)torrent[10]); | |||
item.FinishedTime = (long)torrent[11]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What value is FinishedTime if it's still being downloaded?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It will be 0, or more specifically 64-bit integer: 0
.
beeb071
to
7ae8bcd
Compare
… instead of global setting
7ae8bcd
to
d93923f
Compare
FYI, the proper place to store per-item persistent settings is named customs, e.g. a A good and common way to protect items against automation is either the "Ignore commands" flag, or setting "prio=3" (and then using that to filter them out). ruT gets some things "magically" reconfigured by requiring you to call their "init" script as part of the startup configuration. |
Good to know thanks, and yeah I assumed ruT worked something like that. Purely for historic reasons, I'll leave the previous revisions of this PR before it was squashed here. |
Database Migration
NO
Description
Adds the ability for Sonarr v3 to remove seeded torrents from rTorrent. For this to work it requires seeded torrents be in a view called "seeded", but it also doesn't break if the user doesn't implement this in their rTorrent config.
Example in rTorrent config (assuming ratio is enabled):
For those who would like time-based seeding as well (irrelevant to this PR):
This may be ported to Sonarr v2, but I was unable to build it on my linux machine for some reason so I can't test it.
Todos
Issues Fixed or Closed by this PR